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A FACTOR ANALYSIS OF AN INDUSTRIAL 
MERIT RATING SCALE 


EDWIN EWART, 8. E. SEASHORE anp JOSEPH TIFFIN 
Purdue University 


NDUSTRIES using a merit rating scale which includes a 

if number of traits usually assume that the scale separately 

indicates the merit or standing of the employee on each of 

the traits included. Thus if a scale includes items on job per- 

formance, safety, health, and so forth, it is assumed that the 

rating on each of these items represents a separate and specific 
trait of the employee. 

The purpose of the present study has been to analyze statisti- 
eally such a scale as it is now functioning in one industrial 
plant to determine the influence and significance of each trait 
on the final result. 

One question which is often raised by persons preparing 
merit rating scales for use in industry is the number of traits 
which the scale should include. The present study has deter- 
mined by statistical analysis whether one such scale, embracing 
twelve traits, is actually working in this particular instance as 
the constructors of the scale intended it should. In other 
words, the study has determined how many traits actually 
influence the ratings. 

According to the scale in use in a large industrial plant, 
ratings were obtained on twelve traits, as follows: 


1. Safety 7. Industriousness 
2. Knowledge of job 8. Initiative 

3. Versatility 9. Judgment 

4. Accuracy 10. Cooperation 

5. Productivity 11. Personality 

6. Overall job performance 12. Health 
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INDUSTRIAL MERIT RATING SCALE 483 


Each trait was rated on a continuum which contained five 
‘‘landmarks.’’ (See Figure 1.) 


we. mrORMEO OF 4s. | THOROUGH KNOWLEDGE OF 
<nOwS WTTLE asouT THE 1S FOOLY WELL WOFORMEE F OFTAILS RELATIVE TO WIS | PHESENT 408 AWO HELATED 
ANOWS ROUTINE OmY On MIS Wome womn 
! 


UCHREROMGEREEEEEED QERERATERD RRURIAERROREEEREE REE 
Fig. 1. One item from the Merit Rating Scale. 











In most eases, each employee was rated by three supervisors 
and the average of the three taken as the merit rating of the 
employee. In some cases, where three supervisors were not in 
close contact with the employee, only two raters were used. In 
a still smaller number of cases, the ratings were made by only 
one supervisor. 

Ratings were obtained on 1120 men, although for traits 11 
and 12 only 1092 ratings were available. Tetrachoric inter- 
correlations were computed for all twelve traits. Inspection 
of the table of intercorrelations (Table 1) indicates that in 
general the correlations are high. 

The correlation matrix was factor analyzed, using Thurs- 
tone’s centroid method. Three factors were extracted, the 
extraction process being stopped with the third factor since 
Thurstone’s ‘‘phi’’ test was satisfied at that point. Examina- 
tion of the third factor residuals indicated that only one re- 
sidual lay between .11 and .15, thirteen lay between .05 and .10 
inclusive, and fifty-two were .05 or less. 

The plots from the centroid matrix were rotated to the posi- 
tion of best fit. The factor loadings after rotation appear in 
Table 2. With the exception of item No. 4, simple structure 
was obtained. 

Factor I, a general factor, accounts for most of the total 
variance of the scale considered as a whole. In view of the 
extremely high factor loadings on items 6, 5, and 7 (Overall 
job performance, Productivity, and Industriousness) and the 
high communalities of these items, this factor can well be 
termed ‘‘ Ability to do the present job.’’ It seems a safe 
generalization to state that ratings on this one factor or varia- 
ble would be nearly as valuable in predicting worker compe- 
tence as are ratings on all twelve traits. 
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TABLE 2 


Factor Loadings after Rotation 





Factor loadings 





























h** 
I II IIl 

1. Safety 633 245 — .039 462 
2. Knowledge of job ............ 841 337 — .002 821 
3. Versatility nest -795 325 -030 -739 
4, AGCUIRIE  cciseneeencen 826 448 297 971 
5. Productivity ..... paw 913 039 — .042 837 
6. Overall job performance -961 — .080 — .064 934 
7. Industriousness a 906 009 .007 821 
8. Initiative apenas 887 094 252 859 
9. Judgment on 897 265 — .036 876 
10. Cooperation ascniag 881 — .025 042 779 
LL. Persomal ity ceccececcesscoenceenee 815 .009 259 -731 
12. Health 573 — .016 836 1.027 








* Communalities. 


Factor IT, although less significant statistically than Factor 
I, is nevertheless of interest. In view of the size of the popula- 
tion used, the factor loadings on items 2, 3, and 4 (Knowledge 
of job, Versatility, and Accuracy) may be considered fairly 
significant. However, to name the factor represented by this 
configuration is more hazardous than the naming of the first 
factor extracted. It is suggested, as a possibility, that this 
factor represents knowledge or skill possessed over and above 
the requirements for the specific job. The relatively low load- 
ings on this factor may be accounted for by the fact that the 
‘*factor’’ objectively represented may be inadequately mea- 
sured by the items in the present scale. At any rate, Factor II 
does not contribute to the predictive value of the present scale 
to any such degree as does Factor I. 

Factor III contains only one significant loading, namely, in 
the variable ‘‘Health.’’ Previous studies of the reliability of 
the ratings have indicated that health is rated with a reliability 
lower than the reliability of any other item on the scale. Thus 
the fact that Health stands out as a single factor can be ac- 
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counted for by the relatively low average intercorrelations of 
the Health variable. Furthermore, the health of employees 
would seem to be indicated more accurately by company 
records or medical examination than by supervisors’ ratings. 
This fact, coupled with the fact that Factor III is clearly an 
artifact due to the unreliability of the health ratings, justifies 
the disregarding of Factor III as a real characteristic of the 
employees rated. 

It should be mentioned that Factors I and III are ortho- 
gonal, while Factors I and II are oblique. Thus Factor II, as 
measured by the present scale, is not independent of Factor I, 
the general factor. 

This analysis indicates that for the scale and population con- 
sidered, worker competency could be rated on one, or possibly 
two ‘‘traits,’’ as well as it is now rated on the basis of twelve. 
This in no way contradicts the fact that adequate training of 
raters might enable them to rate workers on more than two 
suggested traits, or that a modification of the scale might re- 
sult, even without more training of the supervisors, in ratings 
which would validly indicate more than two employee traits. 
That is, if the raters were educaied in the general use of rating 
scales, the avoidance of ‘‘halo,’’ and given other training in the 
use of the specific scale in question, or if the items were 
changed so as to include traits not specifically covered by the 
present seale (such as skill apart from performance, or likeli- 
hood of promotion) it is quite possible that a scale of several 
items, applicable to the workers to be rated, could be made to 
function. 

In this regard Stevens and Wonderlic' suggest the rating of 
all men on each trait before going on to other traits. Wads- 
worth? points out that supervisors must be educated to support 
their opinions with facts. Bingham’s position that carefully 
focusing attention on each of several traits probably makes 


18. N. Stevens and E. F. Wonderlic, ‘‘An Effective Revision of the 
Rating Technique,’’ Personnel J., 13, 1934, 125-134. 

2G. W. Wadsworth, ‘‘ Practical Employee Ratings,’’ Personnel J., 1935, 
13, 263-269. 
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the total over-all rating more valid® would seem to be tenable 
only if the traits considered can be shown to be relatively dis- 
erete, and to be associated with the job in question. 

A further argument for the use of merit rating scales cover- 
ing a relatively small number of traits lies in the fact that with 
many traits, the same numerical score for the entire scale can 
result from a number of patterns of merits and demerits on the 
individual traits in the scale.‘ 


SUMMARY 


A factor analysis of the ratings of 1120 men on a twelve trait 
merit rating scale revealed only two factors. Factor I was 
termed ‘‘ Ability to do the present job,’’ and accounted for 
most of the variability of the ratings. Factor II, less signifi- 
cant statistically, possibly represents skill possessed over and 
above the requirements for the specific job. A third factor, 
with a significant loading only in the variable ‘‘ Health,’’ was 
accounted for by the unreliability of the health ratings them- 
selves, and thus can be disregarded as an artifact. 

The implications of the research for industry are that when 
a multiple item rating scale is adopted it is possible that the 
ratings will really reveal only one or two aspects of the em- 
ployee’s merit and not twelve or more aspects as might be 
inferred from casual examination of the scale. 


3 W. V. Bingham, ‘‘ Halo, Invalid and Valid,’’ J. Appl. Psychol., 23, 
1939, 221-228. 


4G. W. Wadsworth, op. cit. 








AN OBJECTIVE TECHNIQUE FOR TESTING 
VOCATIONAL INTERESTS 


DONALD E. SUPER Anp SYLVIA A. ROPER 
Clark University 


THE PROBLEM 


N a recent review (7) of studies in the measurement of 
interests published since 1931, a total of 72 different stud- 
ies involving the use of interest inventories was found, 

while only 2 which had attempted to measure interests by 
more objective methods were located. The former group of 
studies revolved around 7 different inventories, of which 5 
have been published; the latter group of investigations dealt 
with 2 tests, neither of which has been published. These fig- 
ures reveal an important trend in the research of the past ten 
years. Clearly, most investigators have considered the in- 
ventory method the more promising approach, 

The interest inventory technique, first carefully investi- 
gated in the years immediately after the first World War, 
consists of comparing a person’s estimated likes and dislikes 
with those of men who are successful in an occupation, by 
means of a statistical weighting of responses to a large number 
of statements of likes and dislikes. Strong’s Vocational Inter- 
est Blank is the best-known measure of this type (6). 

Objective tests of interest, as Fryer has called them (3), 
might better be called tests of manifest interests, thus differen- 
tiating between the objective treatment of data as in an in- 
ventory or test, and subjective or objective methods of obtain- 
ing the data. The self-report technique of the inventory is 
thus a subjective method of obtaining data, and the measure- 
ment of information, attention, or the ability to perform some 
task is an objective method of obtaining data. 
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Two types of tests of manifest interests have been developed. 
The first of these involves measuring information previously 
acquired about a subject, on the assumption that special inter- 
est in a topic will result in the acquisition of more knowledge 
concerning it than concerning other topics, or in more knowl- 
edge of a certain type than is acquired by other less interested 
persons. O’Rourke’s Mechanical Aptitude Test is sometimes 
cited to illustrate this principle (3), but the newer Cooper- 
ative Contemporary Affairs Test is a better illustration (2). 

The second type of test of manifest interests was developed 
by Wyman (8), who used the association method on the hy- 
pothesis that a child’s associations would reveal his true 
interests. A variation was tried by R. B. Cattell (1), who 
used attention to and memory of pictures in a preliminary 
attempt to measure personality traits. Both methods proved 
promising but were cumbersome to use and have not been 
followed up. Of these various objective tests of interests only 
one new one (2) has survived as an interest test. 

The reason for the concentration of attention on the in- 
ventory method is suggested by this brief glance at methods 
used in measuring interests. One inventory can be used to 
measure interest in a wide variety of activities, and such ques- 
tionnaires may be given to a large number of persons at once, 
the time required rarely exceeding an hour. Tests of manifest 
interests have generally been developed for one field only, are 
cumbersome, and require considerable time, some of them 
being individual tests. 

As many psychologists have pointed out, however, the more 
popular inventory method is not free from defect. Self- 
rating and self-report methods have long been known to be 
affected by bias and by ignorance. Steinmetz (5), for in- 
stance, has shown that college students can raise their scores 
for a given occupation when asked to try to do so. One can- 
not always count on the rapport, intelligence, insight, and 
experience necessary to valid responses. 

Although the inventory method has proved its validity for 
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use in certain situations, a more objective method of measur- 
ing interests, one relatively free from the above defects, is to 
be desired, especially for high school use. After a study of 
previous work, the use of pictures was selected as the most 
promising method and a search was begun for suitable 
material. 


MATERIALS FOR A TEST OF MANIFEST INTERESTS 


Several criteria were kept in mind. First, the pictures must 
represent a wide variety of occupations at and above the semi- 
skilled level and important in vocational guidance. Secondly, 
the pictures must cover various aspects of each occupation 
and give an adequate idea of it to persons not familiar with 
the work. Thirdly, the pictures must be of approximately 
equal quality, informative rather than artistic values being 
desired. 

Materials which seemed to meet these qualifications had been 
collected by the senior author for two or three years when a 
series of filmstrips on vocational education, published by the 
Society for Visual Education, appeared. These were examined 
and seemed to meet the criteria. 

The ultimate plan has been to include in one series of pic- 
tures material presenting the major families of occupations. 
It was, therefore, expected that, if the technique proved satis- 
factory, pictures such as those in the S.V.E. filmstrips would 
be shuffled and combined in one longer filmstrip and interest 
test. Before attempting any such ambitious project, however, 
it was deemed wise to carry out a number of smaller prelimi- 
nary investigations. Several of these will now be described, 
one of them a Master’s thesis by the junior author (4). 

Experimentation was begun with the filmstrip on nursing. 
This film, like the others in the series, consists of about 30 
frames of pictures, each picture followed by a frame with a 
brief statement of occupational opportunities or requirements. 
A group of 13 graduate nurses and supervisors, to whom the 
filmstrip was shown, felt that it conveyed an adequate idea of 
their occupation. After preliminary tryouts it was decided to 
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expose each frame for 15 seconds. The film was to be shown 
after the following verbal instructions had been given: ‘‘ We 
have a film to show you, entitled ‘Nursing As a Career.’ Please 
pay close attention to the film and do not discuss it or any- 
thing else. No questions will be answered.’’ After the film 
was shown, test blanks, to be described directly, were distrib- 
uted with instructions to read the directions and to proceed 
accordingly. No time limit was set, but instructions were given 
to work rapidly. 
SELECTION OF TEST ITEMS 


The first question on which information was desired had to 
do with the type of item which would best test memory for 
content of pictures and facts presented in the filmstrip. An 
analysis of these suggested five parts for the test. The first 
was a multiple-choice test covering items of information made 
familiar by the film, but which the subject might have acquired 
from other sources. A sample will illustrate: 


Nurses’ training usually lasts: 1) one year, 2) two years, 
3) three years, 4) four years, 5) five years. 1 2 3 4 5 


The second part contained descriptions of pictures which 
might have been used in the film: some did appear whereas 
others did not. Distinctions were made on details which were 
in some way related to nursing ; for example, if a picture were 
shown of a nurse attending a patient who had a bandaged 
head, it was assumed that a really interested person would 
notice that it was a head that was bandaged, not a leg, and 
would therefore check the description of a picture mention- 
ing a bandaged head but not one mentioning a leg. 

The third part was a matching test. Column A was a list of 
types of nurses, such as private duty nurse, while Column B 
was a list of duties and qualifications. 

Part IV resembled Part II in that the subject was asked to 
check items which appeared in the film; it differed in that it 
dealt not with pictures but with subjects required for admis- 
sion to nursing schools, studied while in training, ete. Part 
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V was like Part IV, but dealt with reasons for failing in train- 
ing schools. 

To determine the validity of these different types of items, 
two approaches were used. The first of these involved deter- 
mining the correlations of the four larger and differing parts 
of the Interest Test with existing tests known to be of some 
value in selecting nurses, namely, the Otis Test of Mental 
Ability and Strong’s Vocational Interest Blank for Women. 
There were no differences between the coefficients for the parts 
of the Interest Test. The second method was a comparison of 
the mean scores of nurses and high school girls on each part of 
the test. In this case Parts I, II, and III were shown to be 
superior to Parts IV and V, the differences between the groups 
being significant for the former but not for the latter (4). 

These findings leave certain aspects of our question unan- 
swered because of differences in the types of items in these 
three parts. They involve, it will be remembered, retention of 
facts and recognition of pictures presented in the film. The 
latter type of item could be answered correctly only on the 
basis of knowledge gained by seeing the film, or by chance; 
the former could be answered correctly also on the basis of 
general knowledge previously acquired through other sources. 
To minimize this tendency, the directions stated explicitly that 
all answers must agree with the facts given in the film in order 
to be right. But this was not considered enough to rule out 
the effect of previous information. 

In an attempt to determine the role of previously acquired 
knowledge in this test, it was administered to a fairly homoge- 
neous group of 32 high school girls, 16 of whom were randomly 
selected to take the test without seeing the filmstrip, while the 
other 16 were shown the film immediately before being tested. 
In each group of 16 there were 3 girls who were considering 
becoming nurses. The median score of Group I, which saw the 
film, was 86; for Group II, which did not, it was 31. The semi- 
interquartile ranges were 24 and 11. There was no overlap- 
ping of scores, the lowest for Group I being 77, and the highest 
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for Group II being 48. The difference is clearly significant 
despite the small number of subjects. The average score re- 
sulting from previously acquired knowledge being only one- 
third of that resulting from long-term and recent memory, and 
the interquartile range of Group II being less than half that 
of Group I, it was concluded that differences in test scores 
could sefely be attributed to differences in immediate memory. 
Differences between the parts showed that Part II was least 
affected by previous knowledge and that future work should 
draw heavily on this type of item. 

A third problem had to do with the reliability of the method. 
The odd-even reliability coefficient was determined for the test 
as a whole: corrected, this was .83, sufficiently high for present 
purposes. 

VALIDITY OF THE METHOD 


The most important problem was, of course, the validity of 
the method. Five criteria were selected as most suitable: 
marks in training school, ratings by supervisors, intelligence, 
an established measure of interest in nursing, and the ability 
of the test to distinguish between persons engaged in the pro- 
fession and persons not engaged in it. 

The marks of 29 graduate and senior student nurses who 
had trained in a local hospital training school and who had also 
taken the Test of Interest in Nursing were obtained. An ex- 
amination of these showed a very limited variability and made 
their validity as criteria doubtful. These marks were treated 
in three ways for each nurse: (1) all marks were averaged; 
(2) the mean for professional courses only was obtained ; and 
(3) grades in four courses which showed the best distribution 
of marks were averaged. These mean grades were correlated 
with intelligence, Strong’s scores, and scores on the Test of 
Interest in Nursing. The correlation between the mean for all 
marks and the last-named test was .16; no other coefficient 
exceeded .07. 

Ratings by supervisors were available for the 25 student 
nurses. These ratings were filled out each month according 
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TABLE 1 


Correlations between Tests and Criteria 














Criteria Otis Strong Super 
All marks 05 .00 16 
Professional subjects ........... 06 01 07 
Special subjects cceccccssnconn .06 01 01 
Ratings .08 04 — 15 














to a recently instituted plan, and dealt with 9 personality 
traits, each trait followed by a descriptive statement to be 
rated on a 5-point scale. The ratings for the three months 
preceding the testing were used, an average for each student 
nurse being obtained. Analysis of the data showed that little 
confidence could be placed in these ratings, for the usual rea- 
sons. The obtained correlations of —.15 with the Test of In- 
terest in Nursing, .04 with Strong’s Blank, and .08 with the 
Otis Test are, therefore, not surprising. 

In order that the scores on the Test of Interest in Nursing 
might be related to scores on an intelligence test, the Otis Test 
of Mental Ability had been given with a 20-minute time limit 
to all groups tested. The means and sigmas were found to be 
roughly the same for all groups. The coefficient based on the 
35 nurses was .01; that based on the high school group of 111 
senior girls was .14. Neither of these is a significant correla- 
tion. It was supposed, in planning the investigation, that some 
sort of correction for I.Q. would be needed for the Interest 
Test score; apparently the role of intelligence in this test is so 
small as to make this unnecessary. 

That intelligence plays a negligible part in this memory test 
of interest is encouraging, for the retention of material pre- 
sented in the pictures may then be assumed to be the result of 
interest, of previous knowledge, or of both. As we have al- 
ready seen that the scores on the Interest Test are to a large 
extent dependent on knowledge gained by seeing the film, we 
may assume that interest is the primary factor in the test. 

It now becomes important to determine the relationship be- 








494 DONALD E. SUPER AND SYLVIA A. ROPER 


tween scores on the Interest Test and established measure 
of interest in nursing. Strong’s Women’s Blank was selected 
as the best available gauge of this type. This blank, it should 
be remembered, is not a test, but an inventory which compares 
the subjects’ likes and dislikes with those of women successful 
in various occupations. The correlation between scores on 
Strong’s Nursing Key and the Test of Interest in Nursing was 
—.01 for 35 nurses and .001 for 111 high school seniors. Al- 
though it is questionable whether the two groups should be 
combined as one, the total r equals .003. There is clearly no 
relationship between interest in nursing as measured by 
Strong’s Blank and as measured by the instrument developed 
in these investigations. The significance of this finding will be 
discussed after the final check on the validity of the Interest 
Test has been presented. 

This final check consisted of comparing the scores of the 
various groups in order to determine the differentiating value 
of the measures used. (A somewhat different method was used 
in scoring the Interest Test with these groups, changing the 
raw score values.) This comparison reveals mean Otis I.Q.’s 
of 105.8 for the 35 nurses, 108.9 for the 33 high school girls 
who would like to be nurses, and 108.0 for the 78 girls who 
were not considering nursing as an occupation. The means 
for the same groups, in the same order, on Strong’s nursing 
key, were 60.1, 57.1, and 26.8. The critical ratios show that 
Strong’s Blank distinguishes satisfactorily between nursing 
and non-nursing groups, but not between nurses and would-be 
nurses. When the nurses are compared to the total high school 
group, both interest tests show significant differences. 

When similar comparisons were made for the Test of Inter- 
est in Nursing, the nurses’ mean was found to be 58.5, that of 
the would-be nurses 51.1, and that of the non-nurses 39.2. 
The critical ratios in all three comparisons were significant, 
showing that each group is significantly distinguished from 
the others by this test. The high school seniors who are con- 
sidering entering nursing do not score as high, on the average, 
as do the professional nurses, even though they do score higher 
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TABLE 2 


Differences between Nurses, High School Girls, and High School 
Girls According to Ambitions 








‘Test Nurses eg Would-be Non-nurses 
girls nurses 
Otis Mean .......... 105.8 108.5 108.9 108.0 
Sigma ............. 10.1 10.4 9.9 10.0 
Strong Mean 2... 60.1 43.1 57.1 26.8 
a 21.6 25.5 6.5 23.0 
Super ET 58.5 43.3 51.1 39.2 
a 7.3 15.6 12.0 15.0 
I iiiititititiinsiintinniin 35 111 33 78 
Critical ratios: 
Strong 
et 3.87 .79 7.36 
Would-be nurses ... pate. aS or tent te 11.45 
Super 
FRU OOB  ccccceceectescteseeees 11.70 3.05 9.21 


Would-be nurses ... liom pee ae 4.46 











than those who are not considering nursing. That this is true 
of this test, and not of Strong’s, may be due in part to the 
effect of prior knowledge; as we have, however, seen reasons 
for minimizing the importance of this factor at the high school 
level it seems likely that the difference in the methods may 
play a part. Steinmetz’ data indicate that subjects who think 
they are interested in an occupation may make higher scores 
on Strong’s Blank than they otherwise would, the basic inter- 
est remaining the same. Memory, on the other hand, is prob- 
ably less affected by this superficial aspect of interest. It 
would be logical, therefore, for some girls who are thinking of 
nursing to make higher scores on Strong’s Blank than they 
should, whereas they would not show up so well on the Test 
of Interest in Nursing. 


SUMMARY AND DISCUSSION 


It might be well briefly to review our relevant findings 
before discussing their implications. First, previous knowl- 
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edge of nursing was found to play a minor part in scores on 
the Test of Interest in Nursing. The role of intelligence was 
found to be negligible. This test, and the others used, had no 
relationship to success in nursing school (probably because of 
defective criteria). The two measures of interests were quite 
unrelated. Finally, in spite of this last fact, both interest 
tests were found to distinguish between nurses and non-nurses, 
the Interest Test better than the Interest Blank. 

It seems difficult, at first thought, to reconcile these last two 
findings. It might be argued that, since the two tests differ- 
entiate between the two groups, there should be at least a low 
positive correlation between the two measures. The data show 
that there is no relationship, either in the homogeneous or in 
the heterogeneous groups. An analogy may help to explain 
this. We know that psychologists are distinguished from the 
general population both by their superior scores on Strong’s 
Blank and by their superior scores on intelligence tests. 
Strong has reported in the Manual to his blank (6) that there 
is a correlation of .38 between scores on the psychologists’ 
scale of his Inventory and on the Thorndike Intelligence Test. 
This is the type of evidence which would lead one to expect 
the two interest tests of this study to yield a low positive cor- 
relation, for we see here two tests which distinguish a group 
from the general population with just such a relationship. If 
we examine other data of Strong’s we soon see, however, that 
such a conclusion is not warranted in spite of its apparent 
theoretical soundness. Y.M.C.A. secretaries, for instance, are 
known to be of above average intelligence, as they are profes- 
sional workers with college degrees, and they are, of course, 
also distinguished from the general population by their high 
scores on Strong’s Y.M.C.A. secretaries’ scale; but the corre- 
lation between scores on this key and on the Thorndike Test is 
—.18. This holds true for a number of other occupational 
groups and keys. The Thorndike Test and Strong’s Blank 
measure two different types of traits and abilities, and the fact 
that some groups score high on both does not necessarily indi- 
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cate a relationship between the two. Similarly, with our data, 
certain groups are distinguished from the general population 
by their scores on two tests: it is submitted that there need 
not, because of this, be any relationship between the two tests. 
The facts substantiate this hypothesis. 

The above suggests that the Test of Interest in Nursing is 
as valid as Strong’s Blank, in spite of the lack of relationship 
between the two. The question arises, then, as to the meaning 
of this lack of relationship. The answer seems to lie, again, 
in the essential differences of method. If we may assume that 
a subject has with complete truth and objectivity answered 
each item on Strong’s Blank, we then have not a measure of 
the extent of his interest in a given occupation, but, as Strong 
and others have made clear, a measure of the similarity of his 
interests to those of persons who have succeeded in that occu- 
pation. There is very little material in any interest inventory 
that deals with the actual processes and problems of an occu- 
pation; in fact, it is one of the advantages of the Inventory 
that it measures vocational interest patterns by means of likes 
and dislikes which are not themselves strictly vocational. The 
Test of Interest in Nursing, on the other hand, deals only with 
matters pertaining directly to the occupation in question, and 
the degree of interest in these is measured by retention of this 
material. It is, therefore, suggested that, whereas Strong’s 
Blank measures similarity of interests to those of persons en- 
gaged in an occupation, the Test of Interest in Nursing mea- 
sures the degree of interest in that occupation, and that the 
two are not the same. 

Other studies are being undertaken to check the validity of 
these tentative conclusions, in the expectation that a practical 
instrument will result which will supplement the inventory 
approach in an important way. 
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PROCEDURES FOR EVALUATING 
VOCATIONAL CRITERIA 


ROGER M. BELLOWS* 
University of Maryland 


I 


FFORTS designed to evaluate predictive instruments 
3 have generally neglected the fact that basic criteria are 
fallible. Procedures for checking the goodness of the 
vocational criterion, considered as a definition of levels of job 
performance, have been omitted from the scattered reports on 
this subject. Many of these stress the necessity for criteria 
in employee evaluation and as a prerequisite to test develop- 
ment. In addition to the importance of an adequate criterion 
for these purposes the evaluation of programs for orientation, 
personnel selection or classification, or methods of training— 
in fact any research situation that makes use of experimental 
and control groups—demands the use of a ‘‘good’’ criterion. 
First a few specific sources of criterion contamination, and 
procedures for detecting their influence, are to be examined, 
then the deceptive nature of criteria is to be emphasized. Crit- 
icism of the traditional way of interpreting validity which 
assumption of criterion infallibility has engendered is implied. 
Aside from the implications this discussion may have for in- 
terpretation of results derived from investigations it is hoped 
that the procedures recommended may serve as a check list for 
use by those who plan to pretest their criterion before using it 
as the basis for their conclusions. 


II 
1. Contamination by illicit use of predictor information. 
Reported coefficients of correlation or other indices of alleged 


* The writer is indebted to John G. Jenkins and members of a seminar 
on criteria at the University of Maryland for advice and suggestions. 
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validity of predictors are often contaminated by illicit use of 
predictor information. This may be defined as any influence 
of knowledge of predictor information on the part of those who 
determine or contribute to the determination of criterion scores 
of subjects of the investigation upon criterion data, and as a 
result upon indices of validity of predictors. 

Instances of this in the field of education and training are 
numerous. Knowledge of percentiles on the psychological ex- 
amination by instructors responsible for grading has contrib- 
uted to the size of validity coefficients reported at certain uni- 
versity clinics. In one university the instructors made routine 
use of such percentiles when deciding on the students’ final 
grade for the course. These same data were used for evalua- 
tion of external consistency of the test. 

Influence by the spurious use of predictor data is not pos- 
sible if the predictor data are collected after the criterion data 
have been assembled. It is usually negligible when objective 
criteria, ¢.g., work units per unit of time, are employed, but 
even here it may under some circumstances be extremely sig- 
nificant. It is of great importance when the criteria are sub- 
jective, e.g., qualitative efficiency ratings by supervisors who 
have access to the predictor information. It is an even greater 
source of error in validity indices when predictor information 
is used overtly to determine standing on the criterion, ¢.g., 
when the criterion is promotion and the test data are used by 
promoting officers as the basis for making promotions, and 
when this fact is unknown or disregarded by the investigator. 

In cases in which objective criteria are collected in follow-up 
studies of validity of predictor items the criterion data may be 
contaminated because of expectancy motivation resulting from 
supervisor’s knowledge of predictor information. Trainers 
and supervisors are naturally likely to expect more from work- 
ers who are known to have high standing on predictors assumed 
by them to be valid. They tend to motivate the ‘‘better’’ 
workers in a more effective way than the ‘‘poorer’’ workers. 
Motivational influences have been shown to act upon produc- 
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tion’ in indirect and subtle ways, such as informal social or- 
ganizations within a given plant, suggesting the importance of 
expectancy motivation as a possible basis of contamination. 
Particularly when comparing the mean production of a 
group of test-selected workers with that of a group of workers 
selected by the interview the influence of predictor connection 
may contaminate the results. In some situations it is not im- 
probable that supervisors who are eager for their group to 
make a good showing in production will favor the workers 
they think are ‘‘better’’ by giving them, for example, pre- 
ferred positions in the work room, or more efficient equipment. 
Many investigations deal with data collected by a method 
of ‘‘following back’’ to evaluate the predictor-criterion rela- 
tion. In such cases there is a gap of information—the investi- 
gator does not know or disregards the extent to which the pre- 
dictor data have been overtly used to arrive at criterion scores. 
In long-time ‘‘follow back’’ studies, such as those reported by 
Ball? and by Gifford* the time of collection of predictor data 
is separated from the time of collection of criterion data by a 
long interval—in the case of Ball’s study this interval was, 
for one group, 18 years; for Gifford’s study it ranged as high 
as 30 years. Neither of these studies took account of the extent 
to which overt use of predictor data influenced the criterion. 
In such cases the investigator has little opportunity to check 
on overt use of predictor data; he likely has no means of know- 
ing whether the promoting officers did or did not use the pre- 
dictor information as a basis of promotion. In the case of 
Ball’s study the best statement one can make is that overt use 
of it was unlikely ; in Gifford’s the use of it was very likely, so 
that the measures of relation between the predictor and cri- 
terion variables reflect to an unknown degree the use of the 


1 Roethlisberger, F. J., and Dixon, W. J. Management and the Worker. 
Cambridge: Harvard University Press, 1939. Pp. xxii+615. 

2 Ball, Richard 8., ‘‘The Predictability of Occupational Level from 
Intelligence.’’ J. Consulting Psychol., 1938, 2, 184-186. 

8 Gifford, Walter 8S., ‘‘Does Business Want Scholars?’’ Harpers 
Magazine, 1928, 156, 669-674. 
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predictor information by officers of the organization who were 
responsible for promotion. 

Thus Gifford’s conclusion that men from the highest tenth 
of college classes have four times the chance of those from the 
lowest third to stand in the highest tenth salary group does 
not necessarily mean that high achievement in scholarship 
was in itself conducive to high eventual salary, but probably 
that those who have had marks indicating high scholarship 
tend to be more frequently promoted on the basis of such 
marks. The obtained index of relation may be evidence only of 
illicit use of predictor data. 

In the case of Ball’s finding of a Pearson coefficient of cor- 
relation of .71 + .04 between a measure:nent of intelligence 
and occupational status (the index for which was obtained by 
the Barr scale) 18 years later, the cautious reader is justified 
in inquiring whether employers knew of the intelligence scores, 
whether they were used in letters of recommendation from the 
schools, or whether any use whatever of the test scores might 
have influenced the size of the index of relation.* 

Procedures for guarding against this kind of contamination 
are simple ; they consist of keeping predictor information away 
from these who are in any way, even remotely, concerned with 
determining criterion scores. A quotation from the report on 
psychological examining in the United States Army illustrates 
the point—‘‘Before the results of the psychological examina- 
tion were reported, company commanders were asked to rate 
their men.’” 

4 The writer has communicated with C. M. Louttit, who directed Ball’s 
study, regarding the possibility of predictor connection. Louttit indi- 
eated there was no possibility of it in this case, the test records having 
been kept in private files during the 17-year period; he agrees that state- 
ments of possible spurious use of predictor information are essential in 
the reports of investigations of this kind. 

5 Yerkes, Robert M. (Editor). Psychological Examining in the United 
States Army. Memoirs of the National Academy of Sciences, Vol. XV. 
Washington, D. C.: Government Printing Office, 1921. P. 453. 
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2. Contamination by artificial limitation of production.® 
Contamination by artificial limitation of production is defined 
as influence of any constant physical conditions at work which 
determine or contribute to the relative output recorded for in- 
dividual workers. Such conditions operate to increase or de- 
crease relative production of certain workers, or to decrease 
absolute production of some workers. Examples are poor 
lighting conditions in one part of the work room, differences 
in efficiency of machinery or equipment used by the workers, 
and differences in positions on the sales floor or area in the case 
of criteria of sales performance. Piece workers who are paid 
on the basis of the amount produced are often careful to report 
to the work supervisor or foreman any condition that might 
result in artificial limits. Other workers are not likely to be 
as conscientious in reporting such conditions. 

These conditions may influence both objective or subjective 
criteria. In the case of objective criteria this source of error 
may contribute directly to criterion scores, and in some in- 
stances in which the job output is paced mainly by machine, 
artificial limits may be the sole determinant of the criterion 
scores. Subjective evaluations of workers are subject to con- 
tamination from this source of error in direct proportion to 
its presence and to the use made of the resulting production 
data by the rater. 

If criterion data are influenced in this way it is obvious that 
extremely high reliability coefficients would be obtained when 
reliability of the production criterion is checked by odd-even 
periods or by use of successive criterion periods, and that such 
indices of reliability would be spurious since they would reflect 
only the presence of constant artificial limits, and hence would 
be of no value as an aid in estimating the efficacy of the 
criteria. It is also apparent that psychological tests or pre- 


6 This source of error in criterion data, and procedures for guarding 
against it, have been stressed by Jay L. Otis in the vocational criterion 
studies of the Occupational Research Program of the U. 8. Employment 
Service. 





—— 








504 ROGER M. BELLOWS 


dictors which reflect characteristics of the experimental group 
of workers would not correlate with such criteria. Criteria 
which show unreasonably high reliability should therefore be 
suspected of contamination by artificial limitations and ob- 
servations should be made by the job analyst to check this 
source of error. This source of error justifies the use of pre- 
dictability by psychological tests as one of the bases of evalu- 
ating criteria. 

3. Contamination by differential influence of experience. In 
aptitude test validation studies criteria are sometimes collected 
from groups of subjects the members of which have had differ- 
ent lengths of training time on the job, and their proficiency 
records may reflect mainly these different degrees of training. 
In other studies the subjects are followed up on the job and 
criteria gathered at some time after the study has been ini- 
tiated ; here differential influences of experience do not operate 
provided the subjects all had the same amount of experience 
at the start or by the time of the criterion period. 

If criterion data are contaminated by training influences 
which affect the various subjects differently the predictors, if 
satisfactory degree of validity is obtained, would tend to be 
determining the presence of this influence rather than the 
presence of those measurable psychological characteristics 
which are related to job proficiency when length of training 
period is constant. Since determination of the latter is the 
goal of predictor development for aiding in selection, counsel- 
ing, or classification, neglect of this source of contamination 
is serious. Psychological aptitude tests should not be expected 
to reflect amount of specific job training the subjects have had; 
proficiency tests on the other hand should be related to this 
variable. Personal data and interview items pertaining to 
length of experience, often included in trial batteries of pre- 
dictors, should of course show correlation with length of 
experience. 

Marcus’ was one of the first to report’a study for which sta- 


7 Marcus, L., ‘‘Study of Selective Tests for Hollerith Machine Opera- 
tors.’’ J. Appl. Psychol., 1920, 4, 196-201. 














— 





EVALUATING VOCATIONAL CRITERIA 505 


tistical adjustment was made for the influence of experience. 
He gave tests to a small group of card punch machine operators 
who were doing comparable work and on whom production and 
error records were obtained. He used a combination of pro- 
duction adjusted for errors and also for training influences, 
since his various subjects had different lengths of training time 
at the time of the criterion period. 

If the experimental conditions do not permit the selection of 
subjects for the study that have equal lengths of experience 
it is desirable, as in the study of Marcus, to adjust criterion 
scores for this influence. A practical method for correcting 
criterion scores for experience is outlined in a recent pub- 
lication.*® 

If such influences are present in small degree, the criterion 
data may or may not show high statistical reliability. If such 
influences are the main determinants of criterion scores the 
index of reliability should be high, particularly if the cri- 
terion period is short, for example, when one criterion week 
is correlated with another criterion week, because those who 
are untrained and for that reason low in criterion scores for 
one period would also be relative low in the next period. 


Ill 


A few specific sources of contamination of criteria have been 
exemplified. An attempt has been made to point to ways of 
protecting criterion data from these influences which are likely 
to be present. In the following statements six possible checks 
on the goodness of the criterion are discussed and evaluated. 
The first of these is statistical reliability. 

1. Statistical reliability. This check is important as an ini- 
tial step in criterion analysis but when used by itself it has 
serious limitations. Indices of reliability of criterion data 
seem as important as reliability of test data, yet few investiga- 

8 Stead, W. H., Shartle, C. L., and Associates. Occupational Counsel- 


ing Techniques: Their Development and Application. New York: Ameri- 
ean Book Co., 1940. Appendix VII, pp. 258-260. 
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tors appear to have made use of this as a possible basis for 
criterion evaluation. No personnel manager would knowingly 
use criteria of extremely low reliability for making practical 
decisions on employee promotion or separation, yet few users 
of such data are aware of their degree of reliability. 

An example of criteria of extremely low reliability may be 
mentioned. At the start of an investigation by the writer in 
a plant of 123,000 employees on methods of selecting machine 
shop apprentices it was found that, when available subjective 
criteria were split into odd and even criterion periods, the data 
yielded such a low reliability index that it was obvious the 
ratings were worthless for any purpose. Since it was not 
feasible to collect additional criterion data it was necessary 
to abandon the study. 

Reliability as a basis for evaluating may serve as an aid in 
checking both subjective ratings and objective criteria. In the 
former case it is practicable to correlate periodic ratings, or 
ratings of two or more raters or to employ both measures as 
checks. These procedures were used in a study of cafeteria 
employees in which an attempt was made to develop a work- 
able rating device of the specific instance type. By this means 
unreliable items in the scale, items on which different raters 
did not agree, were eliminated, and the device was gradually 
improved with respect to reliability. In the case of objective 
records such as production control data the odd-even periods 
technique may be employed. For example, Spearman-Brown 
correlations ranging from .89 + .02 for the error criterion 
to .95 + .01 for the production criterion, obtained by cor- 
relating odd-even criterion periods on various samples of 
clerical workers have been reported.® 

It should be stressed that the reliability check is an internal 
one and does not insure that the criterion is valid. There is 
no way to validate the criterion objectively since there is no 


* Stead, W. H., Shartle, C. L., and Associates. Occupational Counsel- 
ing Techniques: Their Development and Application. New York: Ameri- 
ean Book Co., 1940. Pp. 66-71, 141. 
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objective, external basis for use as a standard with which it 
may be compared. An unreliable criterion cannot be satis- 
factory ; a reliable criterion may or may not be valid. Data 
pertaining to levels of proficiency which show Spearman- 
Brown reliability of about .85 or above may be considered arbi- 
trarily*® to be adequate in this respect, and may have some 
utility if those of the other procedures for their evaluation 
that are applicable to the problem have been employed and 
favorable results have been obtained. A criterion of high 
reliability may not be satisfactory for practical use if it is 
contaminated by illicit use of predictor data, artificial limita- 
tions of production, or differential experience influences. 
Further, if a criterion is unrelated to other possible criteria, 
if it is not predictable, if it is not acceptable to the job analyst 
or the sponsor, or if its use does not produce a practical change 
in the situation, it is not a suitable criterion regardless of the 
obtained index of reliability. 

2. Correlation with other criteria. Of several single criteria 
equally acceptable on the basis of reliability and other possible 
bases of evaluation, that one which correlates highest with 
other acceptable measures of levels of performance is probably 
best. If, for example, it is found that number of errors made 
is not related to sales performance in terms of amount sold, 
per cent selling cost, bonus earned, returns, amount per sale 
or any other practical criterion of selling proficiency, then the 
error measure would not be satisfactory by itself regardless of 
its reliability because it would not be realistic or representative 
of the whole job. If errors are considered of sufficient impor- 
tance on the basis of other checks they could possibly be pre- 
dicted separately as a secondary criterion and not as the basic 
criterion. 

10 The highest possible validity coefficient that can be obtained with a 
predictor which has perfect reliability is limited by r ab, the coefficient 
of reliability of the criterion: for criterion reliability of .20 the highest 
possible validity of a predictor of perfect reliability is .45; for criterion 


reliability of .80 the highest possible validity is .89. Guilford, J. P. 
Psychometric Methods. New York: McGraw Hill Co., 1936. P. 423. 
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This check also is insufficient by itself for criterion evalua- 
tion; it must be employed in connection with other checks. 
Further, this check places the burden of establishing a stand- 
ard of evaluation upon other criterion measures for which a 
standard of an external type is lacking. If results from this 
check are favorable it is not indicated therefore that the adop- 
tion of the criterion will assure a rigidly scientific standard 
against which the investigator may evaluate predictive instru- 
ments or methods of training. 

3. Predictability. In deciding between several criteria 
which are judged to be equally satisfactory on the basis of the 
internal and subjective checks or upon which the other checks 
cannot be made, it not infrequently happens that the criterion 
that is most predictable by psychological tests and personal 
data items is adopted. The investigator may conclude that 
criteria which cannot be predicted by tests and items found 
to be ‘‘valid’’ for other somewhat similar situations must be 
unsatisfactory. 

If a criterion is contaminated by spurious use of predictor 
data it will of course be highly related to predictor scores; if 
it is predictable to some degree it will have some degree of re- 
liability ; if it is contaminated by artificial limitations of pro- 
duction it will not be predictable by either aptitude or pro- 
ficiency tests; if an experience influence is present in great 
degree aptitude tests will not predict it to any great extent but 
proficiency tests should be highly correlated with it. 

The fact that investigators sometimes resort to predictability 
as a basis for criterion evaluation further emphasizes the diffi- 
culty inherent in the problem of criterion evaluation. Whereas 
in test validation the criterion is used as the external standard, 
in criterion evaluation there is no external standard except 
the predictors themselves. If an external basis is to be used 
it is necessary for the investigator to reverse himself, as fre- 
quently happens, and be content to check his standard against 
that which he is evaluating. The circle in which he must place 
himself is unfortunate. 
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It should be noted also that this criterion check is not avail- 
able for the group on which the derived instrument is actually 
used for practical purposes of selection. It can be made in 
the experimental situation only. 

4. Acceptability to the job analyst. It is customary in the 
evaluation of psychological predictors used in personnel selec- 
tion for the investigator to analyze the job for purposes of 
determining the most promising tryout predictor battery. The 
job analyst may also make an analytical study of possible 
criteria. 

Before collecting criterion data the job analyst is able to 
gather general information pertaining to employee evaluation. 
He may study the situation with reference to production output 
measures and the availability of ratings. He may determine 
whether output, in the case of criteria consisting of produc- 
tion records, is largely individual, or determined by team work, 
or paced by machine. He may further inquire whether there 
are factors such as artificial limitations of production or re- 
lated influences on measures of output such as regulations of 
labor unions which limit the amount of daily production. He 
may check comparability of work of each person on the job to 
insure that all are doing the same job operations and that 
their output is recorded in the same units of production. He 
may check for influence of training and experience on the out- 
put of workers, and if this factor is present he may determine 
whether all subjects of the sample have had the same or differ- 
ent amounts of experience. He also has opportunity to study 
the possibility of influence by illicit use of predictor infor- 
mation. 

His judgment as to the suitability of any of the available 
criteria, or the selection of one from a number of possible ones, 
may be obtained. It is feasible to use rating procedures such 
as the method of paired comparisons to aid in determining his 
opinion regarding the best criterion for use. 

This check is necessarily subjective. After securing the job 
analyst’s opinion it is reasonable to suspect that the criterion 
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adopted may be fallible because his observations may have been 
faulty and his advice unsound. His subjective evaluation cer- 
tainly does not provide an objective standard with which the 
criterion itself may be compared, and unless the study is made 
on a job in which the criterion problem is extremely simple 
and clear-cut, this check will also have little value if used by 
itself. 

5. Acceptability to the sponsor of the study. In the usual 
course of events the sponsor of the study determines to a large 
extent the criterion that is used. He has roughly defined the 
criterion problem and has established a criterion that seems to 
him practical and satisfactory. In selection test studies he 
desires, at least at the start, that his own criterion be used. 
His criterion is considered ‘‘practical’’ and the investigator 
may feel that the goal is to predict this criterion as well as 
possible whether it is good or bad; often he makes no effort to 
evaluate or improve it. 

After checking the sponsor’s criterion with reference to some 
or all of the items outlined above the investigator may find it 
either suitable or unsuitable. If it appears suitable the sponsor 
and investigator are in agreement; if it is not a good criterion 
on the basis of available evidence the only courses left open 
to the investigator are to convince the sponsor that the newly 
developed criterion is better than the one formerly used by 
him, to attempt to predict the poor criterion as required by 
the sponsor, or to discontinue the study. If several criteria 
appear equally adequate to the investigator the sponsor’s 
evaluation could be used as final. 

In addition to the aspect of subjectivity of the sponsor’s 
rating of criteria his evaluation is likely to be biased in favor 
of his former methods; he usually resists change in routine 
procedures for employee evaluation. Further, since he is un- 
trained in psychological method this source of evaluative in- 
formation is likely to be the weakest and should never be used 
in any case without supplementary evidence from other 
sources. 
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6. Production of a practical change in the situation by use 
of the derived instrument. If, by use of a particular criterion, 
predictors have been developed that are judged useful over a 
long period of time, and more useful than the former methods 
employed, then the criterion used may be said to be a practical 
one. It has produced a practical change in the situation. 

Many test programs designed for use in private industry 
have been abandoned after a year or two. This would suggest 
that the criterion used originally was not adequate or that 
there was a change in the situation resulting in decreased value 
of the derived instrument, implying decreased value of the 
original criterion. Continued practical use of the tests derived 
would on the other hand seem to suggest that the criterion 
used was adequate and stable. The matter of practical change 
is usually evaluated subjectively by the sponsor, or such judg- 
ments may be based on group production and output records, 
but in either case other conditions are likely to cut across and 
influence decisions regarding efficacy of the instrument. The 
fact that so many test programs have been abandoned by man- 
agement would suggest that the criteria used have not been 
adequate ; or if adequate at the start of the program, work and 
worker situations have changed to such an extent that the cri- 
terion originally used for ‘‘validation’’ lost its usefulness be- 
cause of such changes. 

Production of a practical change will not help make the ini- 
tial decision regarding goodness of the criterion since this 
source of information is not available at the start of the pro- 
gram, but only after the crucial decisions have been made. 


IV 


No investigator has ever recommended any psychological in- 
strument of known validity for practical use. It is impossible 
to know the future validity of a predictor. 

Only the last source of evaluative information mentioned has 
implications for evaluation of continuing usefulness or sta- 
bility to criteria. When criteria consist of subjective em- 
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ployee evaluation both job and rater changes reduce original 
criterion efficiency and when objective output or error records 
are used job situation changes may operate to reduce effi- 
ciency of the original criterion. 

A predictor designed for a particular purpose is checked 
against a criterion but the criterion used in the practical situ- 
ation is not necessarily the same as that used in validation. 
The job title may remain the same but the job may change. If 
the requirements of the job change, the predictor used is of 
unknown value. There is no technical method for checking 
on such situation changes until the changes have come into 
existence. 

Further, predictors may work well in predicting a criterion 
for one situation and for one population, but not necessarily 
for the same situation with a differently constituted popula- 
tion. If linearity is found between predictor scores and cri- 
terion scores for one population it is not permissible to con- 
clude that arrays of these two sets of scores will be linear for 
a second population. 

Two assumptions regarding future use after validation must 
therefore always be made before recommending an instrument : 
no changes in job and criterion situation, and an identically 
constituted population. No check is available on these as- 
sumptions until after the instrument has been used. The 
investigator in presenting indices of validity of his instrument 
can therefore never justifiably use the present or future tenses. 
He may say ‘‘ Validity was .55.’’ He may not, as is usual, say 
‘*Validity is .55’’ or ‘‘The test will have a validity of .55 with 
the criterion.’’ 

The fact that recommendations for use of tests are based on 
assumptions (sometimes not justified) and the fact that eri- 
teria are always fallible (usually the degree of fallibity is un- 
known) help to account for the decrease in correlations ob- 
tained when a test, found to have a reasonable predictive 
efficiency in one situation, is tried in another situation. These 
facts also help to explain the failure of many test programs 
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established for practical situations and eventually discon- 
tinued by their sponsors because they did not produce practical 
results. 


v 


In conclusion it may be said that no single basis or technique 
for criterion evaluation is adequate, and that each separate 
source of information for evaluation must result in specula- 
tion. These facts have serious implications for traditional 
concepts of test validity. They suggest that size of obtained 
indices of test validity are largely determined by degree of 
fallibility of the criterion, and that interpretation of such 
indices must depend upon knowledge of the criterion employed 
in validation. The criterion may be evaluated clinically by aid 
of the items of information discussed, and this procedure may 
lead to refinements in criteria and in the tests derived from 
their use. 














OPINIONAIRE CORRELATES OF VOCATIONAL 
PREFERENCE FACTORS. II 


THEO. F. LENTZ anv E. F. NICKEL 
Washington University 


HE two articles, of which this is the second to appear, on 
this subject by the same authors report explorations re- 
garding the relationships between vocational preferences 

and other personality manifestations, more particularly those 
resulting from expressions of opinion on social and personal 
issues. Whereas the preceding article reported a successful 
search for relationships between opinionaire reactions and 
specific vocational preferences, the present article will report 
the results of attempts to relate opinionaire reactions to groups 
of vocational preferences.* 

For this purpose three of the seven factors into which Thur- 
stone groups the vocations in his Vocational Interest Schedule 
were selected, namely, academic, biological, and descriptive. 
The page of 129 vocations in the Youth Expressionaire in- 
cludes seventy of the vocations in the Thurstone Schedule, only 
one less than in the complete instrument.? On the basis of his 
reactions to these seventy items each person in the population 
studied was given a score for each factor. This scoring was 
done in accordance with the standard instructions for scoring 
the Thurstone Schedule, two points being given for each ‘‘L’’ 
reaction to an item included in the factor group, one point for 
each ‘‘I’’ reaction, and no eredit for a ‘‘D’’ reaction. In 


1In the first article, ‘‘Opinionaire Correlates of Specific Vocational 
Preferences,’’ appearing in the August issue of this journal, will be found 
a more complete description of the source of the data used in both articles. 

2 The vocation appearing in Thurstone’s Schedule but not used for this 
study is ‘‘Cattle Rancher.’’ This could affect only the athletic factor 
which is not being considered in the present study. 
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other words, the data secured by this process were practically 
the same as those which would have been secured by adminis- 
tering the Thurstone Schedule to the population studied. 

For each factor the preference reaction scores of four hun- 
dred high school graduates were studied as follows: The one 
hundred high score persons on the factor were contrasted with 
the one hundred low score persons on each of 480 items of 
opinion by noting the percentage of each of the two groups 
reacting positively to the item. The results are illustrated for 
the three factors in Tables 1, 2, and 3, respectively. 


TABLE 1 
Opinionaire Differences between Those Having the Highest Scores and 


Those Having the Lowest Scores for the 
ACADEMIC FACTOR 














Per cent affirm. 
resp. by Sie 
High Some cent Item 
scale score aie. 
persons persons 

67 35 32 Our present system of athletics in America is 
at fault in that it does not provide for mass 
participation. 

51 22 29 I would be willing to go to a social dance at- 
tended by both white and negro students. 

74 56 28 Women should take a deep interest in polities. 

72 48 24 Are you philosophically inclined? 

71 49 22 Do you often have a feeling of inability to 
cope with the situation? 

47 64 -17 Are you greatly annoyed at being pushed 
around in a crowd? 

45 63 -18 Without the incentive of financial gain little 
would be accomplished. 

43 64 —21 Generally speaking, Americans are more in- 
telligent and enterprising than the people of 
most any other country. 

37 67 -30 The people who complain most about unem- 
ployment wouldn’t take a job if you gave it 
to them. 


19 51 -—32 I dislike radicals of any kind. 
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Table 1 presents items illustrative of the differences found 
between those persons who had the highest scores for the aca- 
demic factor, which includes such vocations as college profes- 
sor, historian, sociologist, etc., and those who had the lowest 
scores for this factor. In this table, as in Tables 2 and 3, the 
first column shows the percentage of the high score people mak- 
ing an affirmative reaction to the item; the second column, the 
percentage of the low score people making an affirmative reac- 
tion to the item, and the third column, the percentage differ- 
ence between the two groups. 

Considering the first item in the table, ‘‘Our present system 
of athletics in America is at fault in that it does not provide 
for mass participation,’’ it is seen that 67 per cent of the people 
having the highest scores for the academic factor responded 
affirmatively and 35 per cent of the people having the lowest 
scores for that factor responded affirmatively, giving a differ- 
ence of 32 per cent. 

Inspection of the illustrative items in this table indicates 
that the high score academic people are less conservative, more 
tolerant, more inclined to feel that women should have inter- 
ests outside their homes, and have less race prejudice than the 
low score people. Other items not listed indicate that the 
former are more at ease in social situations and less impulsive 
in thought and in act than are the latter. As in the case of the 
illustrations for the specific vocations in the first article, the 
indications mentioned are strengthened by differences found 
on many other unlisted items. 

Table 2 illustrates the differences found between those re- 
ceiving a high score for the descriptive factor, which includes 
such occupations as novelist, journalist, philosopher, ete., and 
those receiving a low score. 

Consideration of these items and others correlated with this 
factor shows that the high score descriptive people are more 
inclined to jealousy, radicalism, self-confidence, and idealism, 
and less inclined to race prejudice than are the low score 
people. 
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TABLE 2 


Opinionaire Differences between Those Having the Highest Scores and 
Those Having the Lowest Scores for the 
DESCRIPTIVE FACTOR 














Per cent affirm. 
resp. by Per 

High ren cent Item 
pa score ait. 

persons persons 

59 36 23 Do you enjoy taking part in an argument 
merely for the sake of arguing? 

69 46 23 Do you often feel that you could do things 
better than the leader of your group? 

53 32 21 A man who refuses his soldier’s bonus because 
he regrets his participation in the World 
War elicits your admiration. 

36 17 19 Are you often jealous of other people? 

50 32 18 Radical agitators and propagandists should 
be allowed to speak publicly in parks and 
streets. 

29 43 -14 A doctor is never justified in helping to end 
another’s life even when the person asks 
that he do so. 

13 32 -19 Are you more interested in athletics than in 
intellectual things? 

24 44 -20 Would you mind working with a negro? 

32 53 -21 As long as our captains of industry are as 


humane to their employees and as long as 
wealthy people are as philanthropic as at 
present, there will be no need for socialism. 

32 59 -27 In social conversations are you usually a lis- 
tener rather than a talker? 





Illustrations of the differences between those with the high- 
est scores for the biological factor, which includes such voca- 
tions as biologist, chemist, surgeon, etc., and those with the 
lowest scores for this factor are found in Table 3. 

Tendencies indicated by these items include, on the part of 
those having high scores for the factor, more scientific minded- 
ness, and consequently less inclination to believe popular mis- 
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TABLE 3 
Opinionaire Differences between Those Having the Highest Scores and 
Those Having the Lowest Scores for the 
BIOLOGICAL FACTOR 











Per cent affirm. 
resp. by Per 
High Low ‘ent Thom 
coe score aiff. 
persons persons 

73 42 31 Our national government should appropriate 
20 billion dollars for research work (chiefly 
in the social sciences). 

72 47 25 Do you enjoy working with animals? 

40 16 24 One is justified in neglecting his health if it 
is necessary to do so to accomplish very 
important work. 

66 47 19 Grain found in Egyptian tombs frequently 
sprouts after 3000 years of dormancy. 

46 31 15 Do you feel angry when someone laughs at 
your mistakes? 

12 34 -—22 Would you rather have $10,000 than a college 
education ? 

32 53 -21 In ease of imminent danger a porcupine pro- 
tects himself by shooting his quills. 

22 43 -21 Trial by jury has been, and always will be, 
the most effective way of securing justice. 

27 44 -17 Do you sometimes feel afraid to face the 
future? 





conceptions, more susceptibility to ridicule, more regard for 
formal education, and less race prejudice than on the part of 
those having low scores for the factor. Particularly interest- 
ing is the number of items, not listed, which show that the 
low score persons much more often admit fear of things or 
situations. 

Table 4 attempts to present a more statistical picture of the 
results obtained on the basis of comparisons of high and low 
score people for the various factors. The percentage of 480 
opinionaire items, as well as the number, which were found to 
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differentiate significantly the high and low score groups for 
each factor are presented in this table. 


TABLE 4 
Summary of Significantly Differentiating Items 
(For Vocational Factors) 














Contrasted groups "Sites been si 
No. Per cent 
High Score Cases versus Low Score Cases: 
Academic Factor 67 13.9 
Biological Factor 42 8.7 
SINE SPI iss cerreremiecsetitnnetie 64 13.3 
Average 57.6 11.97 








* Items showing a difference of more than twice the standard error of 
the difference are considered as significantly differentiating. 


It is interesting to note that the average yield of signifi- 
eantly differentiating items for the factors, 11.97, is very little 
higher than the average yield of items for the specific voca- 
tions, which, as mentioned in the preceding article, was 11.17 
per cent. 


THE THURSTONE FACTORS AND THEIR OPINIONAIRE BATTERIES 


Having discovered relationships between vocational factors 
and items of opinion, a further step was taken to determine to 
what extent the findings might be the result of chance. This 
consisted of correlational procedure similar to that reported 
in the first article for the specific vocations. It will be recalled 
that Tables 1, 2, and 3 illustrated the items which show signifi- 
cant relationships with the factors studied. For each factor 
these correlated items were grouped into an ‘‘opinionaire bat- 
tery’’ for purposes of scoring. Each person in the population 
was given an opinionaire battery score for each factor or ‘‘vo- 
cational battery,’’ receiving one point each time he agreed with 
an item showing a larger proportion of affirmative answers on 
the part of the high vocational battery score group, and an- 
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other point each time he disagreed with an item showing a 
larger proportion of affirmative answers on the part of the low 
vocational battery score group. 

Table 5 shows the reliabilities of these vocational batteries 
(factors) and of their opinionaire batteries, and the intercor- 
relations between each vocational battery and the opinionaire 
battery derived from it as described above. 


TABLE 5 


Vocational Batteries (Factors) and Their Opinionaire Batteries 
Reliabilities and Intercorrelations 














Odd-even r Vocational 
, battery 
Vocational versus 
battery Vocational Opinionaire opinionaire 
(factor) battery battery battery 
Uneorr. Corr. Uneorr. Corr. Uncorr. Corr. 
Academie .....ccceoo 55 71 72 84 43 56 
Biological _.................... 83 91 .39 56 57 80 
Descriptive 0.0... 61 -76 64 .78 52 68 





In the first column of Table 5 the reliabilities of the scores 
given for each factor in accordance with the scoring procedure 
prescribed by Thurstone are indicated. The second column 
shows these reliabilities corrected by application of the Spear- 
man-Brown formula. In the third column appear the relia- 
bilities of the opinionaire batteries derived from each voca- 
tional battery or factor. In the fourth column these same 
reliabilities as corrected by the Spearman-Brown formula are 
shown. In the fifth column are the uncorrected intercorrela- 
tions between the vocational battery score and the opinionaire 
battery scores; the sixth column shows these same intercorre- 
lations corrected for attenuation due to the unreliability of 
the original measures. 

Consideration of these data reveals the fact that in the case 
of the academic and descriptive factors, the opinionaire battery 
has greater reliability than the vocational battery, whereas the 
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biological factor battery has much greater reliability than its 
opinionaire battery. In every case, however, the opinionaire 
battery shows a significantly high correlation with the voca- 
tional or factor battery from which it was drawn and further 
evidences the relationship between vocational preferences and 
attitudes and opinions. 

In connection with the batteries derived from both factors 
and items, it has been mentioned previously that the factor 
contrasts yielded on the average no greater number of signifi- 
cantly differentiating items than did the specific vocation con- 
trasts. The data in Table 5 indicate that the average uncor- 
rected intercorrelation between factor battery and opinionaire 
battery, .51, is the same as the average uncorrected intercor- 
relation between item and battery (see Table 5 in preceding 
article), and that with respect to the intercorrelations corrected 
for attenuation, the average between item and battery, .79, is 
considerably higher than the average between factor battery 
and opinionaire battery, i.e., .68. From this it might be con- 
cluded that the determination of attitude and opinion corre- 
lates for a specific vocational preference is no less feasible, and 
possibly more so, than the determination of similar correlates 
for a factor or group of vocational preferences, 


SUMMARY 


In the first of these two articles correlations were presented 
which showed evidence of significant relationships between 
reactions to opinionaire items and preferences for specific vo- 
eations. The correlational data presented in this second article 
substantiate and elaborate this evidence in terms of indicated 
relationships between expressions of opinion and vocational 
preferences based on groups of vocations, i.e., the Thurstone 
factors. As was true in the case of specific vocations, the num- 
ber of individual opinionaire items discovered which showed 
significant correlation with groups of vocations or factors was 
far greater than might be expected by chance. From these 
correlated items test batteries were developed having predicted 
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reliabilities ranging from .56 to .84, and showing correlations 
with the factors from which they were derived ranging from 
.56 to .80 (estimated). 

CONCLUSIONS 


On the basis of the evidence presented in these two articles, 
the following conclusions, mentioned briefly in the first article, 
are drawn: 

1. Opinionaire items of both a social and a personal nature 
show distinct relationships with vocational preferences. These 
relationships, especially those involving social opinions, can be 
considered as indicating additional areas from which to choose 
vocational correlates. If this transfers to other criteria of 
vocational potentiality, it should furnish the means for ampli- 
fying our predictive instrumentation. 

2. Over a period of time these additional correlates or types 
of correlates should improve our insight into the psychology 
of vocational preference and help to answer the question as to 
what kind of person is attracted by what kind of vocational 
label, as well as what are the types of validities and invalidities 
which inhere therein. 

3. These inter-relations among vocational preferences and 
other personality manifestations yield further evidence that 
vocational preference is a function of one’s character and 
hence useful for character measurement. 

4. Since this wider area of personality manifestation is re- 
lated to vocational preference, and since, in other studies, vo- 
cational preference appears to be related to vocational practice 
and vocational fitness, it seems probable that this type of per- 
sonality variable is significantly related to vocational practice 
and to potential vocational success. 

The nature of this relationship, however, represents the out- 
standing question unanswered in this paper. To what extent 
are vocational preference correlates identical with vocational 
practice correlates? Follow-ups on this line are contemplated 
by the authors and other members of the Character Research 
Institute. Contemplations around this question frequently 
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raise the question as to whether in some respects the fact of 
vocational preference may not have a more true relation to 
vocational potentiality than does vocational practice. 

The testing power of opinionaire items as over against voca- 
tional preference items has not been adequately tested by the 
research here reported. It looks as though the vocational 
items might be more powerful for vocational guidance pur- 
poses. However, the opinionaire items may measure some- 
thing more general and possibly something therefore more 
stable. 








THE DISTRACTING EFFECT OF NEARBY 
CARTOONS ON THE ATTENTION 
HOLDING POWER OF AD- 
VERTISEMENTS 


JOHN J. MCNAMARA anp JOSEPH TIFFIN 
Purdue University 


N a recent series of experiments using the Purdue Eye 
| Camera’ it has been possible to evaluate objectively the 
effect of cartoons adjacent to advertisements upon the at- 
tention holding power of these advertisements. The Purdue 
Eye Camera makes it possible to photograph the time spent 
upon each advertisement (or specific part of the advertise- 
ment) by a reader leafing through a magazine. The camera is 
so constructed that the magazine is placed unopened on an 
easel before the reader. The reader voluntarily opens the 
magazine and leafs through the pages at whatever rate he 
wishes, stopping for as long, or short, a time on each item of 
interest as he desires. The camera is driven by an electric 
motor which is activated by a silent switch so that it can be 
started and stopped without awareness on the part of the 
reader. By means of this arrangement, only eye movements 
occurring on the material being studied need be photographed. 
The reader is told that an investigation is being made of what 
interests him in the magazine. While he realizes that it is 
some sort of experimental situation, he is not actually aware at 
the time that his eye movements are being recorded. 
In the present series of experiments, of which this report 
covers only one part, various groups of men and women read- 
1J. 8. Karslake, ‘‘The Purdue Eye-Camera: A Practical Apparatus for 


Studying the Attention Value of Advertisements,’’ J. Appl. Psychol, 
XXIV, 4, 417-440, 1940. 
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ers from an industrial city were asked to leaf through a recent 
issue of the Saturday Evening Post. This magazine was ob- 
tained two days prior to newsstand reiease. The readers, 
therefore, had not had an opportunity to see the magazine 
prior to the experiment. The readers used in the study were 
divided into four groups. One group leafed through a stand- 
ard issue of the magazine exactly as it appeared later on the 
newsstands. The other three groups leafed through issues 
which had been ‘‘doctored’’ as indicated below: 


Group 1. Magazine unaltered. 

Group 2. Magazine altered by blocking out cartoons near 
certain advertisements with printed material simi- 
lar to the copy appearing around the cartoon. 

Group 3. Magazine altered by interchanging certain adver- 
tisements and cartoons from left to right page (or 
vice versa). 

Group 4. Magazine altered by blocking out cartoons near 
certain advertisements and changing advertise- 
ments from left to right page (or vice versa). 


The reversal of left and right hand pages was not essential 
in studying the effect of cartoons, but was necessary for cer- 
tain other parts of the investigation which are to be published 
at a later date. 

One hundred and twenty readers were used in the experi- 
ment, but since each reader was allowed to see a given adver- 
tisement in only one of the four situations mentioned above, 
no single group contained more than 35 readers. 

The results of the measurements are shown in Table 1. This 
table reveals that of the ten situations studied, nine show a 
definitely greater average time spent by the reader upon the 
advertisement when the adjacent cartoon had been eliminated 
than when the cartoon was present as it appeared in the nor- 
mal issue of the magazine. The only exception to this finding 
is the ninth item in the table, which shows a slight and statis- 
tically insignificant reversal. The averages show that for the 
five advertisements studied in ten positions, the average time 
spent with the nearby cartoon present was 2.94 seconds, 
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whereas when the cartoon had been eliminated this average 
was raised to 4.54 seconds. This difference of 1.60 seconds, or 
54%, is 4.4 times its standard error and hence is significant 
from a statistical viewpoint. 

It should be emphasized that this study deals only with the 
effect of cartoons upon nearby advertisements when all other 
factors are held constant. It is likely—indeed, perhaps certain 
when one considers the reader preference for cartoons—that a 
scattering of cartoons throughout the advertising section of a 
magazine increases the reader ‘‘traffic’’ through this section. 
This increase in traffic probably results in a net gain for the 
the advertisements considered as a group. But when the 
traffic is held constant (as in this experiment) those adver- 
tisements which happen to be on a spread which does not 
contain a cartoon seem to be in the preferred positions. 

















THE PLACE OF THE BERNREUTER PER- 
SONALITY, STENQUIST MECHANICAL 
APTITUDE, AND THURSTONE VO- 
CATIONAL INTEREST TESTS IN 
COLLEGE ENTRANCE TESTS 


HOMER B. REED 
Ft. Hays Kansas State College 


of the Bernreuter Personality Inventory, the Stenquist 

Mechanical Aptitude Test, Part I, and the Thurstone 
Vocational Interest Schedule in a team of college entrance tests 
most of which were tests of scholastic achievement. We wished 
to discover the extent to which the Bernreuter scores were 
related (1) to scholastic achievement, (2) to mechanical apti- 
tude, (3) to vocational interests, (4) to ratings of traits of the 
same name given by teachers well acquainted with the pupils 
rated, (5) the relation of the Stenquist test to scholastic 
achievement and to vocational interests, and (6) the relation 
of the Thurstone test to scholastic achievement and to me- 
chanical aptitude. 

The subjects used consisted of 259 freshmen entering Fort 
Hays Kansas State College the fall of 1938, and 266 high 
school students living in the vicinity of the college. The col- 
lege freshmen were given the Bernreuter Personality Inven- 
tory, the Thurstone Vocational Interest Schedule, the Sten- 
quist Test of Mechanical Aptitude, Part I, and nine scholastic 
tests which yielded achievement scores in Humanities or His- 
tory of civilization, English composition, English literature, 
social science, physical science, total achievement, average of 
high school grades, and college aptitude. The high school stu- 
dents were also given the Bernreuter, Thurstone, and the Sten- 
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Ske purpose of the investigation was to find out the place 
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quist tests, but in the scholastic field they were given the Iowa 
High School Content Examination. In this investigation the 
high-school students were used only for the correlation of 
measured traits and interests with estimated traits and 
interests. 

The essential procedure of the investigation was the correla- 
tion technique. Sources were obtained for twenty-two traits 
of the college students and all the intercorrelations calculated. 
The results are given in Table 1. 

The scores on the Bernreuter Personality Inventory have 
little or no relation to scholastic achievement. The correla- 
tions of the scores on four Bernreuter traits—Neurotic Ten- 
dency, Self Sufficiency, Self Confidence, and Sociability—with 
the achievement tests vary from —.151 to +.196 and have an 
average of .076 P.E. + .04 when this average is calculated 
regardless of sign. The highest correlations are barely out- 
side the limits of error and this average is only twice the 
probable error. When the correlations between the Bern- 
reuter scores and total achievement are studied we see more 
clearly the lack of an important relation between them. Total 
achievement correlates — .096 + .042 with Neurotic Tendency, 
13 + .042 with Self Sufficiency, —.107 + .042 with Self Con- 
fidence, and .019 + .043 with Sociability, all of which are 
within the limit of error. Although the relationship is slight, 
we see that what there is is in agreement with common sense. 
According to expectations, Neurotic Tendency has a slight 
negative relationship with scholastic achievement, Self Suffi- 
ciency a slight positive relationship, lack of Self Confidence 
a slight negative relationship, and lack of Sociability a slight 
positive relationship. 

The Bernreuter scores have little or no relationship with 
mechanical aptitude as this is measured by the Stenquist test. 
Neurotic Tendency correlates .088 + .042 with Mechanical 
Aptitude, Self Sufficiency correlates .185 + .041, lack of Self 
Confidence correlates .059 + .043, and lack of sociability corre- 
lates .173 + .041. Only two of these are large enough to be 
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on the border of the limits of error. It thus appears that 
Neurotic Tendency and lack of Self Confidence have no rela- 
tionship to mechanical aptitude while Self Sufficiency and 
lack of Sociability have a very slight positive relationship 
with it. 

The Bernreuter scores have little or no relationship with 
vocational interests as these are measured by the Thurstone 
Voeational Interest Schedule. The average of twenty-eight 
correlations between four Bernreuter traits and seven voca- 
tional interest fields is .057 + .042. The lowest correlation in 
this group is —.003 + .048, between Neurotic Tendency and 
interest in Athletic Activities, and the highest is .127 + .042 
between lack of Sociability and interest in Physical Science. 
Even the highest correlation in this group is only three times 
the size of the probable error. 

The Bernreuter scores have little or no relationship with 
ratings of traits by the same name as they are given by teachers 
well acquainted with the pupils rated. In the investigation of 
this relationship 266 high school pupils were given the Bern- 
reuter and other tests. Before the tests were scored, each 
pupil was rated by each of four teachers on the Fort Hays 
Rating Scale, which consists of thirty-six traits to be rated on 
a seale of five, the degrees, 1, 3, and 5 of each trait being defined 
by descriptive adjectives. Neurotic tendency was determined 
by the average score made on three traits: quickness of excita- 
bility, intensity of excitability, and stability of emotion. The 
traits Sociability, Submission, Dominance and Introversion- 
Extroversion had the same names in the rating scale as in the 
Bernreuter Inventory. The rating score in all cases was the 
average of the ratings given by four judges. The correlation 
between the paired scores and ratings obtained by Miss Carol 
Bryan, one of the writer’s students, was .005 + .042 for 
Neurotic Tendency, .025 + .044 for Sociability, .090 + .042 for 
Submission-Dominance, and .083 + .042 for Introversion-Ex- 
troversion. These contrast rather sharply with the correla- 
tions found in a similar manner between the paired scores and 
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ratings for intelligence and for mechanical ability, which were 
.619 + .025, and .613 + .026, respectively. The lack of corre- 
spondence in scores and ratings of the personality traits 
may be due to a variety of factors, subjectivity of the 
ratings, difficulty of a common understanding of the meaning 
of the personality trait, and lack of validity in either the Per- 
sonality Inventory or the rating scales or both. 

So far, our search to find significant relationship between 
Bernreuter scores and scores in other tests administered to 
our freshmen has been disappointing. By accident we did, 
however, discover that they were related to certain types of 
behavior not measured by our team of tests. One of the 
writer’s students, Miss Stella Crawford, became interested in 
the personality differences between doodlers and non-doodlers 
as these are defined in ‘‘Mr. Deeds Goes to Town.’’ She com- 
pared the Bernreuter scores of almost fifty doodlers with those 
of about an equal number of non-doodlers. She found signifi- 
eant differences between these two groups in Introversion and 
in Dominance. Doodlers are less introverted than non-dood- 
lers but much more dominant. They are also less neurotic, 
more self sufficient, more self confident, and more sociable, 
but these differences are not statistically significant. 

The Stenquist Mechanical Aptitude Test, Part I, has for the 
most part, insignificant correlations with the scholastic achieve- 
ment. The average of ten correlations between this test and 
ten scholastic achievement tests is .054 + .04 when this is cal- 
culated regardless of sign. There is, however, a significant 
negative correlation (—.262 + .039) between Mechanical Apti- 
tudeand English composition and a significant positive corre- 
lation (.231 + .041) between Mechanical Aptitude and Intel- 
ligence. Its chief value in a group of freshmen entrance tests 
is to discover ability in non-academic and mechanical lines. 

The Stenquist Mechanical Aptitude Test, Part I, has few 
significant correlations with the seven fields of vocational inter- 
est measured by the Thurstone Vocational Interest Schedules. 
The average of these correlations when calculated regardless 
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of sign is .149 + .04. Two of the correlations, however, appear 
significant ; that between Mechanical Aptitude and interest in 
Physical Science, which is .410 + .036, and that between 
Mechanical Aptitude and interest in Athletics, which is 
.234 + .041. 

The Thurstone Vocational Interest Schedule has few signifi- 
cant correlations with tests of scholastic achievement. The 
average of seventy such correlations when calculated regard- 
less of sign is .107 + .04. Interest in descriptive activities 
has a fair correlation (.216 + .04) with achievement in English 
literature. 

Interest in Academic Activities has a fair correlation 
(.247 + .04) with average high-school grade and also with col- 
lege aptitude (.218 + .041), and interest in Athletic activities 
has a fair correlation (.254 + .04) with achievement in physi- 
cal science, but none of these is high enough to serve as a basis 
for guidance into specific courses. 

Although the Thurstone Vocational Interest Schedule has, 
on the whole, insignificant correlations with scholastic achieve- 
ment, Bernreuter personality traits and mechanical aptitude, 
the fields of interest claimed to be measured by it correlate 
pretty well with each other. These correlations range all the 
way from .244 to .656, which shows that their lines of relation- 
ship are far from orthogonal. 


SUMMARY 


We set out to find the place of the Bernreuter Personality 
Inventory, Stenquist Mechanical Aptitude Test, Part I, and 
the Thurstone Vocational Interest Schedule in a team of college 
entrance tests most of which were tests of scholastic achieve- 
ment. We find that the Bernreuter scores have little or no 
relation to scholastic achievement, mechanical aptitude, voca- 
tional interests, or to the ratings given by teachers on traits 
of the same name. We find, however, that two of the tests 
diserminate fairly well between persons classified as doodlers 
and non-doodlers. Our conclusion is that the Bernreuter 
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Personality Inventory does not provide a sufficient basis for 
guidance in regard to choice of college courses or of a voca- 
tion. Its usefulness for other purposes was not investigated in 
this research. 

The scores on the Stenquist Mechanical Aptitude Test, Part 
I, also have little relation to scholastic achievement or to Bern- 
reuter personality traits, but the test is useful for discovering 
talent along non-academic and mechanical lines. 

The Thurstone Vocational Interest Schedule reveals few 
significant relations with scholastic achievement or with Bern- 
reuter personality traits. It appears not to provide a sufficient 
basis for guidance in the choice of college courses, but how 
useful it is for guiding people into suitable vocations was not 
investigated in this research. 





A PRACTICAL AID TO OCCUPATIONAL 
ORIENTATION 


G. A. WALLAR! 
Scranton-Keystone Junior College 


VAILABLE information on occupational choices,’ 
A points to the conclusion that students overlook and 
fail to consider some of the rational bases for occu- 
pational choice. There is need for the provision of logical 
bases on which students may make their occupational choices. 


1 The work reported here is one aspect of research completed in partial 
fulfillment for the degree of Doctor of Philosophy, received at the Ohio 
State University, August, 1940. The writer wishes to thank Drs. Harold 
E. Burtt, Sidney L. Pressey, and Harold A. Edgerton for their active 
cooperation in making this study. 

2 Surveys of occupational choices of students are found in the follow- 
ing references: 
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—a Sixteen-Year Span,’’ Sch. Rev., 1936, 44, 689-693. 
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376-384. 

Douglass, A. A., ‘‘ Vocational Interests of High School Seniors,’’ Sch. 
and Soc., 1922, 16, 79-84. 
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on the Vocational Choice of High School Boys,’’ J. Appl. Psychol., 1935, 
19, 203-212. 
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Sparling, E. J., ‘‘Do College Students Choose Vocations Wisely?’’ 
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among Technical School Pupils,’’ Human Factor, 1933, 7, 347-353. 
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More adequate bases for vocational choice may be provided in 
the counseling situation if students (and counselors) overcome 
the tendency to give too much attention simply to interest and 
too little attention to the student’s actual knowledge of the 
work in question, his capacity for that work as he might can- 
didly estimate it, and his probable opportunity to enter the 
work if preparation for it were made. 

This emphasis on interests has been partly the result of the 
many attempts to measure interests. There have been two 
general approaches to the measurement of interests ; one group 
of investigators attempting to measure interests in more or less 
subjective terms while another group has attacked the problem 
indirectly. The general group of studies which attempt to 
measure interest indirectly do not seem compatible with the 
underlying philosophy of our present approach. These mea- 
sures may give the counselor necessary insight but they may 
have no effect on the counselee’s attitude and, since our ap- 
proach is based on the premise that the counselee himself make 
the choice and be convinced of the suitability of his choice, a 
more desirable approach would be one which may affect the 
counselee’s attitude. 

While there are many studies which measure interests sub- 
jectively, none of them has resulted in a counseling instrument 
which gives the counselor clues as to the occupational thinking 
of the subject. Early studies made use of anecdotal methods. 
Then, the questionnaire became popular following Kelley’s 
Interest Questionnaire, but the majority of these studies at- 
_ tempted to differentiate various groups using the techniques 
developed by the intelligence testers. Miner,’ alone, obtained 
information on which to base individual interviews with voca- 
tional counselees. The inventories developed by the Bureau 
of Personnel Research at the Carnegie Institute of Technology 
as amplified by Cowdery and Strong have received the most 
analysis and application. One of the main weaknesses of the 


3 Miner, J. B., ‘‘An Aid to the Analysis of Vocational Interests,’’ J. 
Educ. Res., 1922, 5, 311-323. 
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Strong Vocational Interest Blank* is that the scoring keys 
available are almost entirely on the professional level, i.e., 
there are few jobs for which the Strong blank may be scored 
which do not require at least collegiate training. This would 
almost eliminate any possibility of finding through the Strong 
blank, jobs which might be available to individuals who are 
probably not capable of (or are not interested in) doing col- 
lege work. A second weakness, not inherent in the blank 
itself, is a result of inadequate training on the part of indi- 
viduals who interpret these scores. While Strong himself 
emphasizes the necessity of using these ratings only in con- 
junction with other considerations, many counselors and stu- 
dents interpret them as being much more definite and final 
than the scores are intended to be. A third objection is that 
the Strong Vocational Interest Blank concerns itself with 
interests only. It seems hardly reasonable to tell a boy that 
his interests identify him with lawyers and college professors 
when other evidence points to the belief that he will not be 
capable of completing college. 

The present approach is an attempt to overcome some of the 
weaknesses of these earlier procedures. The purposes of the 
Occupational Orientation Inquiry® are: (1) to enable the stu- 
dent to organize and appraise his occupational thinking; (2) 
to help the counselor obtain relevant information concerning 
the student’s occupational thinking; and (3) to provide a 
schedule for the collection and study of group data concerning 


* Published by Stanford University Press, Stanford University, Cali- 
fornia. 

5 Strong, E. K., Jr., ‘‘Interests of Engineers, a Basis for Vocational 
Guidance,’’ Personnel J., 1928-29, 7, 441-454. 

Strong recognizes this difficulty and says (p. 442): ‘‘In many respects 
it would be far better to tell a boy entering high school that his interests 
identify him with ‘Group C’ occupations, including, for example, civil, 
electrical, mechanical, and mining engineering and chemistry, and pos- 
sibly several others also, than to tell him his interests point toward any 
one of these alone.’’ 

6 Published by The Psychological Corporation, 522 Fifth Ave., New 
York. 
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occupational thinking. This article is a description of the use- 
fulness of the Inquiry in the first two situations. 

The Occupational Orientation Inquiry is divided into three 
parts, which, when viewed as a whole, give a fairly complete 
picture of the individual’s experiences and interests, his own 
ratings on a long list of jobs, and a résumé of his own occupa- 
tional problem. The student begins, following pertinent per- 
sonal information, by reviewing his vocational interests and 
experiences. The following directions are given him: 


‘The Occupational Orientation Inquiry is presented to 
you to help you find yourself vocationally. 

‘* As a first step you are asked to give a brief vocational 
history of yourself in the space below. Include such 
things as the jobs you have held (not only full-time work 
and paying jobs but also part-time and non-paying work 
such as helping your father at his work, or caring for 
children), the age at which you did this work, how long 
you worked at it, and how well you liked it. Be sure to 
describe the kind of work you did rather than giving just 
the name of the job. Include also a brief history of your 
vocational interests at various ages giving the origin or 
bases of each interest.’’ 


This review is designed to help introduce the student to the 
blank in a systematic manner which may remind him of expe- 
riences he has forgotten or deemed unimportant, and to help 
the student to understand that his early experiences are impor- 
tant from a vocational point of view. This résumé also helps 
the counselor judge the reliability of the present interests of 
the student, and gives him a brief background for understand- 
ing the student’s views. 

These experience and interest histories have been found 
valuable in bringing to light information which probably 
would not have been found without this part of the Inquiry. 
It is almost as valuable, if not more valuable, in situations in 
which the subject has had no experience as it is in those rare 
instances in which the subject has held many jobs. Consider- 
able insight may be gained from a consideration of these his- 
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tories. Two illustrations follow. In the urst of these, lack 
of experience or contact with occupations is noteworthy. Al- 
most nothing this young man has done has affected his voca- 
tional thinking. 


‘‘The only work I have done was to help around the 
house, painting and cleaning in particular, although I 
made a few articles of furniture. This work was done 
while I was in grade and high school. House work I de- 
tested, painting I did as a matter of course, but I enjoyed 
constructing. 

‘*From early childhood to 6th or 7th grade I seemed to 
specialize in making cranes but I grew out of that and so 
far have aspired slightly to be a surgeon, lawyer, engineer, 
and aviator. The army has always held my interest, espe- 
cially, the field of flying. I was perhaps influenced in this 
by the army and their flying fields at home. I thought I 
was interested in Electrical Engineering but am not so 
sure now. My father is one and it is about the only engi- 
neering field I have ever come in close contact with. I 
don’t know what I want now.”’ 


The foregoing is certainly in contrast to the history written 
by another 18-year-old boy who has made almost every voca- 
tional experience meaningful to him in his occupational choice. 


‘*My first business venture was instigated by my father 
and consisted of selling magazines. Ever since then I 
have had a dislike for selling. 

‘*From the age of 12 on I assisted the Y. M. C. A. physi- 
cal director in gym and swimming classes. It was during 
this period that I decided to go into teaching physical 
education as a vocation. I worked four summers as coun- 
selor, of swimming especially, at a junior boys’ camp. 
Started this at the age of 14. 

**T have worked at several small jobs from time to time: 
age 17, checker at a municipal swimming pool and for a 
short time, this last summer as a delivery man for 

‘*Ever since that first desire to go into physical educa- 
tion I had had no other thoughts or inclinations until last 
year, when I studied Chemistry in school. The subject 
fascinated me so much that I worked for A’s and obtained 
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them. Out of this grew a less strong desire to become a 
research chemist.”’ 


A counselor reading these statements would already have a 
fair picture of the problems of these boys. 

On the two inside pages of the Occupational Orientation 
Inquiry, where the entire chart may be seen as one display, 
is a carefully compiled list of 224 occupations, ranging from 
those requiring no special training to fields necessitating post- 
collegiate technical specialization. The following directions 
precede the list of jobs. 


‘* All of us know that many times vocational choices are 
based on inadequate reasons. Perhaps there is a passing 
interest in that field or there may be a very good friend 
in that occupation. However, in choosing an occupation 
one should carefully analyze all aspects of the work in 
any job which he might care to choose as his life career. 
Hence, you are now asked to consider the occupations 
listed below from four important standpoints. Look the 
list over carefully and, by means of the scales below, rate 
yourself in each occupation as to your knowledge, interest, 
ability, and your probable opportunity for placement in 
each field.’’ 


The scales used in making the ratings follow: 


KNOWLEDGE 
4. Ihave been regularly employed in the job. 
3. I worked part time or helped in the job. 
2. Ihave general knowledge of job but no knowledge of details. 
1. Ihave less than average knowledge of the job. 
If you have no acquaintance with the job, leave the space blank. 


INTEREST 
4. Ithink I would enjoy it as a life work. 
3. I think I could develop sufficient interest to enjoy it. 
2. Iam indifferent but would work at it if there were the chance. 
1. I would work at it only as a last resort. 
If you would not work at it under any conditions, leave the space blank. 


ABILITY 
4. Ithink I could have exceptional ability in the field. 
3. I could be successful in the job. 
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I would be able to do average work. 

I would find it difficult to work in this field. 

If you would find it practically impossible to make a living at it, leave 
the space blank. 

















OPPORTUNITY FOR PLACEMENT 

4. Iam practically assured a job. 

3. Ihave contact with the field. 

2. Iknow people in the field through whom I might gain contact with the 
job. 

1, Ihave no existing contacts in the job. 

If you have almost no possibility of contact in the job, leave the space 
blank. 


It must be remembered in interpreting the data obtained 
from these ratings that they are subjective self-ratings and are 
not to be considered final evidence in any sense. The ratings 
are estimates on the part of the subjects and, while they are 
not objective, they do reflect the thinking of the individuals 
concerning these aspects of occupations. 

Having made the necessary ratings, the subject evaluates his 
vocational problem in the light of all the above considerations. 
The following suggestions are made: 


‘Now, following consideration of your vocational his- 
tory and these jobs, what vocational advice would you give 
yourself and why? You may find it helpful to go back 
over the check list designating several occupations on 
which you have rated yourself highly and try to evaluate 
in a critical and common sense manner your total possi- 
bilities in each. Consider also such factors as ability to 
finance the training necessary, ability to handle the tech- 
nical part of the job, the health and physical requirements, 
ete. If you have made a tentative vocational choice which 
does not appear on this list or in which you did not rate 
yourself highly, consider it in exactly the same way. This 
evaluation is not to be considered final, but you are to 
evaluate carefully your possibilities in each of several 
occupations. ”’ 


These evaluations also give the counselor considerable in- 
sight into the thinking of the counselee. Some evaluations are 
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almost worthless from the viewpoint of the validity or the in- 
sight into the problem shown by the student. However, even 
these are of help to the counselor. For example, here is an 
evaluation written by a girl who has never held a job, either 
paying or non-paying: 


‘I think that I should try to be a teacher. It is the 
thing that I have always wanted todo. I have been urged 
by teachers and my parents to go ahead in such a course. 
I particularly want to be an elementary school teacher 
because I enjoy being with little children. I have taught 
Sunday School once or twice with children about ten years 
old. As matters stand now I think that I shall be finan- 
cially able to continue my education.”’ 


There is certainly no good evidence of real thinking about 
occupational plans. The counselor in this situation may be 
able to investigate the reasons for this choice and may be able 
to help this girl reach a satisfactory solution. The student, 
also, may recognize that immature reasons have been instru- 
mental in reaching an occupational choice. This may lead to 
a more rational approach to the problem. 

In contrast with the last illustration, the following evalua- 
tion written by a 20-year-old boy who had had approximately 
average vocational experience showed considerable thought 
and planning: 

**1. Actor: This I can eliminate through the fact that 
it is so difficult to break into the field. I would still have 
to take much more training than I now possess. It would 
not, however, give me a chance to use my mental faculties 
as I would like to do, in the sense of reasoning. 

**2. Clerical worker: I can dispense with that quickly. 

I worked at it. It doesn’t take much brain work. Any 

average person could be quite successful. I note also that 

my interest would be practically nil. 

**3. Lawyer and Teacher: Herein lie my major inter- 
ests either within the field of law or teaching, or possibly 
a combination of both. Law was an early interest of mine 
as a youth but I rejected it for an immature reason. 

**4. Salesman: I have gotten a bit of self expression out 
of this work. To me there was quite a thrill in putting 
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over a sale, cracking a hard customer by some particular 
approach or reasoning on my part. From its standpoint 
of no real specialized study of knowledge is the only thing 
that doesn’t appeal to me. If I should fail in attaining 
my goal in these others, I would fall back on this.’’ 





This evaluation would certainly give the counselor several 
possible leads to follow. Of course, the discussion would 
depend upon many other circumstances not presented here, 
but several courses are suggested without recourse either to 
the ratings or to the interest and experience history. 

Several case studies will be presented here to illustrate vari- 
ous situations in which the use of the Inquiry as a whole has 
proved profitable. Each of these studies may be regarded as 
typical and it would be possible to present many similar ones. 
It is not implied, however, that the Inquiry is of value in 
every case. One exceptionally able college freshman asked to 
take the Inquiry after he had heard of its use with a good 
friend of his. After a few minutes’ conversation, he was given 
a copy with the usual suggestions with regard to completing 
it. After he had worked on it for about fifteen minutes, he 
said: ‘‘T don’t think this will do me any good. I’ve thought 
about all these things and many more with regard to all jobs 
in which I’m the least bit interested. I can tell you the out- 
come of my thinking without wasting time on this.’’ Being 
a bit dubious of this statement, the counselor suggested that 
he come in and discuss his plans. He had actually narrowed 
his choice to three groups of possible occupations; he gave 
excellent reasons for considering these and these only; his 
problem was what to do next. The Occupational Orientation 
Inquiry is an attempt to bring the occupational thinking of 
individuals to that point—to just such a systematic considera- 
tion of factors important in occupational choice. 

However, individuals of his foresight are not too frequently 
met even on college campuses—consequently, the need for the 
Inquiry. Five case studies illustrating different uses of the 
Inquiry will be presented : 
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(1) Illustrating the most common use of the Inquiry in 
situations in which the individual has made no decision and, 
in fact, has done little thinking about occupations. 

(2) Illustrating the change from an obviously unsuitable 
choice to a program which may have good opportunities for 
successful adjustment. 

(3) Illustrating the change of occupation from one which 
is probably above the individual’s ability level to one in which 
it is probable that he will succeed. 

(4) Illustrating the usefulness of thinking in terms of 
groups of jobs or fields rather than specific occupations. 

(5) Illustrating the usefulness in self-evaluation without 
special help from a counselor. 


Case #1: Neil was referred to my office by a friend who 
had received some help from how-to-study suggestions given 
him. Neil was a freshman in his second quarter as a student 
in the General Engineering curriculum. His school work had 
been very poor: during his first quarter he had failed algebra 
and engineering drawing, had received a D (the lowest pass- 
ing mark) in chemistry and a C in English. He was very 
discouraged and wanted some suggestions that might improve 
his work. I opened the interview by attempting to find out 
what he planned to do when he graduated from college. He 
wanted to be a salesman in a city. He could give no logical 
reason for choosing the engineering course in college except 
that his world was divided into two categories, agriculture and 
engineering: He had lived on a farm and in a small town all 
his life. He had had enough of that. I suggested that we 
might get the first helpful clues from the Occupational Orien- 
tation Inquiry and asked him to fill it out immediately. 

While he filled out the blank, I checked his grade record and 
his score on the American Council on Education Psychological 
Examination. His grades were as he had reported them; his 
A. C. E. percentile, however, was 66, indicating that he prob- 
ably had ability to do approximately average college work. I 
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also talked with the boy’s faculty adviser who ‘‘had wondered 
why the boy was in engineering.’”’ 

In the interview which followed, considerable information 
was available from the Inquiry. Neil’s interest and experi- 
ence history follows: 


**T have held such jobs as: working on a farm when I 
was of the ages 12, 13, 14, 15, 16, 17 and helped fix a 
bridge. I carried papers 13, 14,15. I worked in a cloth- 
ing store every day after school from 4-6 and Saturdays 
8 a.M. to 10 p.m. and all vacations. I have worked in my 
father’s dairy in the summer and very occasionally when 
smaller and more so now working every Sunday evening 
and most of the summer. I would have liked this work 
but I never had any time off, not even a day in seven weeks 
(this is for two summers straight). I liked my clothing 
store work until last year when I began to tire of it. The 
work was selling behind the counter in dairy and clothing 
store. I have helped my Dad on special deliveries and 
holiday stand where we sold ice cream and I liked this.’’ 


This write-up would certainly make any counselor wonder 
why the individual was enrolled in engineering! This con- 
fusion became greater as we went over the ratings of the jobs 
listed in the center pages of the Inquiry. Engineer was rated 
(on a seale of 0 to 4) (2) in knowledge, (3) in interest, (2) 
in ability, and (1) in opportunity! There were a great many 
jobs (28 in all) which were rated highly enough to be consid- 
ered possibilities in the first listing of the jobs. Many of these, 
however, were eliminated in the interview which attempted to 
verify the ratings. Most of the jobs eliminated were rated 
highly because of expressed interest based on monetary aspects 
of the jobs, e.g., ‘‘judge’’ was rated highly but on questioning, 
Neil admitted that the only reasons for so doing were the 
esteem with which a judge is held in the community and the 
income that he thought judges received. When the verifica- 
tion of ratings was completed, two groups of occupations 
remained as those in which Neil felt he had reasonable chances 
for success; one group centered around selling or commercial 
matters and the other around dairying. Neil would not even 
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consider the possibility of dairying until it was pointed out to 
him that his objections were based primarily on relatively 
minor aspects of the work and that his experience would give 
him many advantages in competition with others who had not 
been so fortunate. He finally decided that owning and oper- 
ating a dairy would combine the two fields in which he was 
most interested and in which he had the greatest possibilities 
of success. He changed courses to one which combined Dairy 
Industry and Economics and, at present, is making a very 
satisfactory record in that field. 

The Strong Vocational Interest blank was also administered 
in this case. This test gives an excellent picture of Neil’s in- 
terests. The occupations which received scores of A are: 
Production Manager, Policeman, Personnel Manager, Office 
Worker, Purchasing Agent, Sales Manager, Real Estate Sales- 
man, Life Insurance Salesman and Advertising Man. Had the 
counseling been based on considerations of interest alone, it is 
possible, even probable, that no consideration of dairying 
would have been made. The addition of the aspects of knowl- 
edge and opportunity, however, brought out the specific field 
in the commercial area in which the chances for success appear 
greatest. 


Case +2: The case of Russell P. is similar in some respects 
to that of Neil. Russell thought he had made a choice which 
was satisfactory in every way. The use of the Occupational 
Orientation Inquiry was valuable in several ways in this case. 
In the first place, Russell’s choice was determined to a large 
extent by his ‘‘hero-worship’’ for a young man who was suc- 
cessful in the field in question. In the second place, the pro- 
gram which evolved included a rather extended try-out period 
which gave Russell an opportunity to gain some necessary 
experience in his new choice. 

Russell was enrolled as a first year Veterinary Medicine 
student, but toward the end of his first quarter he began to 
doubt the wisdom of his occupational choice. He was doing 
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well in English and chemistry but could not follow the pace 
set by other students in animal husbandry and zoology, which 
were the courses most closely connected with his major field. 
When he called at my office for help in reorienting himself, he 
explained his choice of Veterinary Medicine. He had always 
been interested in horses even though he had always lived in 
a large city. His only experience with animals (aside from 
one summer spent on a farm when he was 7 or 8 years old) 
had been the result of a friendship of a young veterinarian 
who took Russell on calls with him. Russell was very much 
impressed by the ease with which this veterinarian diagnosed 
the case, administered treatment, and walked off with a good 
bit of money for such an easy job. That (and his admiration 
for his friend) was the basis for his choice of Veterinary Medi- 
cine as a profession! 

The vocational and interest history on the first page of the 
Inquiry revealed that clerking in a drug store and soda foun- 
tain was the only experience Russell had had. He claimed 
an early interest in farming, although working with animals 
was the only aspect which interested him. 

The ratings on the jobs on the two inside pages indicate a 
great range of interests and possibilities. Russell had had 
experience in less jobs than the average college freshman, but 
he was interested in more and estimated that he had ability to 
succeed in more occupations than 75 per cent of the college 
freshmen. The following are the jobs which were considered 
in the interview since they had been rated most. highly: 


Occupations K I A 0 
Animal husbandryman. .......... 2 4 3 2 
I INE rcsietcscctoiigenesiverencene 2 4 4 2 
SE tic rehniniinasiatbel 2 3 3 2 
RN OE isc eceiceeen 2 4 3 1 
Creamery operator .................... 2 4 3 1 
ER 2 4 3 1 
I - ailacinnieicentcensisihinipastiens 4 1 2 4 
I iin iccenesintiemndniaatsie a 4 3 3 
BE ET MiltsictanctnSocucccctunicain’ Oe 4 3 1 











548 G. A. WALLAR 











Occupations K I A 0 
Judge 2 4 3 1 
PHOtOgrapher  ncccccccccceicseneensennen 3 + 3 2 
Rancher 2 + 3 2 
Soda fountain clerk ......... 4 0 3 4 
Soldier 2 4 3 1 
Ve i 8 3 2 + 


These jobs do not group themselves except for those centered 
around farming and animals. This group seemed to have the 
strongest possibilities in spite of the fact that Russell had had 
no experience in any of them and was at quite a disadvantage 
in comparison to most of his classmates who had been brought 
up on the farm. Russell’s evaluation of these jobs gives con- 
siderable insight into his viewpoint: 


** Animal husbandry—Everything O.K. except where do 
I get a job? ‘ 

‘* Army officer—Swell, but I can’t go to West Point and 
there is no future in this otherwise. 

‘*Bank teller—I’d like this, but I don’t know anything 
about it and I’m afraid the pay isn’t much. 

‘*City manager—rather an aspiring position, but I like 
planning and directing things and this sounds interesting. 

‘*Creamery operator—Heard enough about it to interest 
me. 

‘‘Dairyman—Very much interest, but I know nothing 
about it or the income you get. 

‘‘Druggist—Don’t care to make prescriptions; other- 
wise O.K. and well prepared. 

‘*Farmer—O.K. but costs too much to get started and 
not much income. 

‘*Fur farmer—Very much interested, but no informa- 
tion about it. 

*‘Judge—Fine, but I don’t like politics or rather not 
the kind we have. 

‘*Photographer—lI like photo taking and know a bit 
about it but you can’t eat film. 

‘‘Rancher—Same as farmer, but on bigger scale. 

*‘Soda fountain clerk—Experienced but I wouldn’t 
want the job. 

‘‘Soldier—No go unless you’re an officer and that’s 
impossible for me. 

‘*Veterinary—Everything O.K., but studies.’’ 
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These evaluations show a considerable lack of knowledge 
about jobs and also bring out one point which was confirmed 
in interviews; he is very much concerned about the monetary 
aspect of any job. The interview immediately following the 
completion of the Inquiry attempted to isolate groups which 
might be considered broad fields on which to base further 
study. This did not prove satisfactory so further reading 
about some of these jobs was suggested. This was supple- 
mented by interviews with men in most of the fields in which 
there seemed to be a possibility of success. This further study 
eliminated from consideration all occupations except animal 
husbandryman and dairyman, the selection of either of which 
would be subject to the earlier objection that he had been 
brought up in a city and had had no opportunity to try his 
hand in these fields. However, one of the men in the Depart- 
ment of Dairy Industry was able to find Russell a job on a 
large dairy farm. Russell kept this job for nine months and 
returned to college thoroughly convinced that he wanted to be 
a dairy husbandryman. He is at present continuing his 
studies in this field. 


Case #3: The case of Robert B. illustrates the usefulness 
of the Inquiry in enabling the individual to change his occu- 
pational choice from one which is probably above his ability 
level to one in which it is probable that he will succeed. The 
choice in the first instance was based on interest alone while 
the subsequent choice was the result of a consideration of 
additional factors. 

Robert came to me for suggestions which might improve his 
academic work in engineering, in which he was a second quar- 
ter freshman. During his first quarter he had failed algebra 
and non-collegiate English and was placed on scholastic proba- 
tion. He wanted to know what was the matter with his study 
habits. After talking with him for some time (during which 
I was convinced that his interest in engineering, particularly 
aviation, was real), I administered several tests which might 
give clues as to weaknesses. Test scores confirmed deficiencies 
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in English and mathematics and showed poor study methods 
particularly in connection with reading. A centile rank on 
the A.C.E. of 37 indicated ability to do college work to be 
slightly below the average college freshman. Suggestions as 
to methods of study were given in several conferences with 
Bob, but they apparently did little good as he barely passed 
his repeat courses. In fact, he received D’s in everything he 
was taking. During his third quarter the conferences were 
continued, at his request, but the emphasis shifted from one 
of study techniques to one of vocational and educational choice. 
Bob almost refused to discuss possibilities of changing plans 
at first, but as the quarter went on, he began to doubt his own 
ability as an engineer. He was asked to fill out the Occupa- 
tional Orientation Inquiry. The following statements ap- 
peared on the first page in answer to the request for a voca- 
tional history : 


‘‘Have delivered flowers, it was fun but that’s all. 
Took tickets at —-———— State Fair. It was tiring work 
but a lot of fun because of the chance to see all the dif- 
ferent people. Have worked for my father in the laundry 
and dry cleaning business. Have worked there in the 
summer time for about the last four summers, also when 
I got out of high school I worked there for about seven 
months. I have worked at about everything except drive 
a truck. This includes advertising, pressing, in wash- 
room, in dry cleaning, making appointments for him with 
salesman, ete.’’ 


This information is certainly significant, especially since it 
was the first knowledge the counselor had had of any voca- 
tional experience. It is interesting to note that there is no 
comment regarding liking or disliking the work in the laundry 
and dry cleaning establishment. It was revealed in a later 
interview that Bob had simply never thought of the possibility 
of entering this business even though his father had a success- 
ful business built up. 

The following brief table shows the rating on knowledge, 
interest, ability and opportunity on the occupations which 
were rated highly enough to be considered as possibilities: 
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Occupations K I A 0 
Advertiser eae 4 2 2 
Be liek epsisesiviepirrssneseeoecens 3 4 4 3 
Building contractor .................... 1 4 3 2 
Buyer of merchandise ............ 2 4 2 0 
Dry cleaner 4 3 4 4 
Employment manager ............. 2 4 3 3 
NIN | jaccipeaieiscichatiashbsenmescies 2 4 2 2 
RE Eee 1 4 4 1 
Factory foreman .......c.... 2 4 3 4 
a S 2 3 2 3 
Te 1 4 3 1 
Office manager .......... wiltlcnenil 2 4 3 3 
_ a ee 1 a 3 1 
Purchasing agent 00000... 2 4 3 4 


After we had listed the jobs rated highly and had been 
discussing and verifying the ratings, Bob suggested that we 
could consider several of them in one group since they were 
rather closely related. He suggested the following groups: 


Advertiser Aviator 

Buyer of merchandise Explorer 

Employment manager 

Office manager Factory foreman 

Purchasing agent Machinist 
Manufacturer 

Building contractor 

Engineer Dry cleaner 

Oil miner 

Aviator 


During the discussion of his possibilities in these groups, 
Bob noticed that each of the possibilities in the ‘‘commercial’’ 
group had been fostered by his experience in his father’s 
business and that each would be satisfied somewhat if he went 
into that business. He decided tentatively, in the light of his 
failure as a student in engineering, to consider seriously the 
possibility of continuing in his father’s employ after his sum- 
mer’s work. (His interest in aviation could easily become a 
hobby from which he could gain considerable enjoyment in an 
avocational manner.) This work was to be tried over a period 
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of a year, and, if at the end of that period further schooling 
was considered necessary, Bob would enroll in a business or 
commerce course which could be directly applicable to his 
new choice. 

A comparison of the results of the Strong Vocational Inter- 
est Blank and the results obtained with the Occupational 
Orientation Inquiry is possible in this case. On the Strong 
blank, three occupations were rated A: Engineer, Production 
manager, and Purchasing agent; four received B ratings: 
Chemist, Farmer, Carpenter, and Printer. There is, thus, 
considerable agreement between the two approaches with the 
exception that dry cleaner and the emphasis on commercial 
occupations was almost lacking in the Strong Vocational In- 
terest Blank. There would be the need for considerably more 
information and probing on the part of the counselor to 
supplement the Strong questionnaire than would be needed 
to make the Occupational Orientation Inquiry practicable. 


Case #4: This case illustrates the usefulness of the Inquiry 
in a situation in which the subject has done little organization 
of his thinking. It also points out an additional advantage to 
the procedure in which groups of occupations are considered 
rather than specific jobs. 

Richard K. called at my office for help in choosing a vocation. 
He had heard that tests were available that would tell just 
what one ought to do—what job one might best fill. After 
explaining the impossibility of such a specific solution, I 
suggested that he might find some value in filling out the 
Inquiry... The purpose and procedure were explained to him 
and he completed the blank in a short while. In the interval 
between his completing the Inquiry and the next interview, 
a check of his record was made. He was found to be doing 
slightly below average college work; his A.C.E. percentile was 
54; there was no indication of any factor which would bar him 
from being at least an average college graduate. 

The interest and occupational history on the first page gave 
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no leads as to possible jobs but was useful in emphasizing 
Richard’s lack of experience. He had never worked except as 
a waiter in his fraternity house. The ratings on the jobs, 
however, did have valuable information to contribute. The 
following occupations were rated highly : 














Occupations EK I A 0 
Animal husbandrymaz ......... 4 3 3 4 
Athletic director ......................... 3 4 3 2 
Auto salesmmam i.e ccccccenn 2 3 2 3 
Coach 3 3 3 2 
Deermomstrator ...cccccccsecocsesocceeeeesee 2 3 3 2 
Farm bureau agent ................... 3 3 2 2 
Farm MAMager on eccceccocineeennne 3 4 3 2 
Farmer 2 3 3 3 
Ra ree Sa ee aE 2 3 3 3 
Hotel manager 0. cece 2 4 3 2 
Real estate salesman ................ 2 4 3 4 
Salesman ar 4 3 4 
Traveling salesman. .................... 2 4 3 3 


In the evaluation on the fourth page of the Inquiry, Rich- 
ard wrote: 


**T think I should be a salesman of some sort, but I don’t 
know what field to enter. I must learn to manage my time 
better if I am to be much of a success. My logical place is 
back on the farm but I am not greatly interested in the 
farm, only the livestock end of farming really interests 
me.’’ 


In this situation no one part of the Inquiry is outstanding in 
value but if all three parts are considered together, one can 
see several possibilities. There seemed to be four groups of 
occupations which were worth considering: a farm group, a 
selling group, an athletic group, and grocer and hotel manager. 
When these groups were pointed out, the comment was to 
eliminate the athletic group as being a hobby and to discount 
grocer and hotel manager as the results of minor ‘‘hero 
worship’’ episodes when Richard was younger. That left two 
groups, farm-centered and selling-centered. The possibility 
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of combining these two was mentioned and greeted with sur- 
prise. Richard had never thought of that and, in fact, could 
think of no occupations which could combine the two. It was 
suggested that he think the matter over for a few days and 
come in again. When he returned he had already made pre- 
paratory moves to change to Agricultural Economics. He saw 
that such a course would enable him to have considerable lee- 
way in making a later, more final choice. This new course 
would give him opportunity to explore several fields in which 
he is interested without the necessity of making a final decision 
immediately. 

The results of the Strong Vocational Interest Blank are 
available in this case also. Three occupations were rated A: 
Sales manager, Real estate salesman, and Life insurance sales- 
man while Social Science teacher and Advertising man were 
rated B. Farmer, which was instrumental in making a more 
definite choice within the selling field, was a non-significant 
score. In general, the two sets of results coincide although 
there is one whole field which is not touched by the Strong 
blank but which plays a very important part in the planning 
of this case. 


Case #5: This case is the most atypical of the case studies 
presented here. Only a few instances have come to the atten- 
tion of the writer in which there is no need for a counselor. 
This is undoubtedly explained, to some extent, by the fact that 
only a small percentage of the student body presents itself for 
aid in choosing a vocation. The majority of those who have 
done considerable thinking about the matter may not need help 
and consequently do not seek any. 

The individual in question had done considerable thinking 
about his future in terms of a life work. He needed most of 
all an opportunity to systematize his thinking and to have the 
endorsement of someone he respected—that is, he needed to 
have someone verify his conclusion. The Occupational Ori- 
entation Inquiry confirmed his belief that he had chosen wisely. 
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David P., an exceptionally able (A.C.E. centile rank 96) 
freshman in the College of Engineering, asked to take a battery 
of tests which might help him decide which branch of engineer- 
ing he might best enter. After we had talked a while, he sug- 
gested that he really was not convinced that he should be in 
engineering although his grades had been considerably above 
average. He had always been somewhat interested in mathe- 
matics and was contemplating a shift to a commerce curric- 
ulum with a major in accounting or actuarial work. I sug- 
gested that the Inquiry might not be of much help in his case 
but he wanted to fill it out. 

His interest and vocational history is exceptionally com- 
plete and informative : 


‘First work—helping mother with housework—helping 
in yard from 7-14 year—no pay, small allowance. Dis- 
liked it very much. 

**Sold magazines—8-14. Small commissions—quit be- 
cause it was hard for me to meet and sell people. 
**Carried papers—14-18. Morning and evening de- 
livery—enjoyed the work very much except for the secur- 
ing of new subscriptions. 

**18-19—for 14 years. Assistant to heating engineer 
of plumbing and heating wholesale house. Work included 
drafting, estimating, design, dictation and general upkeep 
of private office. Enjoyed my 14 years here more than 
any other period of my life, both because of the nature of 
the work and the men I worked with and met in the 
territory. 

**20—3 months manual labor—grain elevator construc- 
tion crew. Hard work—10 hours a day—but enjoyed it. 
‘*While carrying papers wanted to be a ‘big-shot’ on the 
newspaper. Visited editorial rooms—helped district man- 
ager with work. This desire, of course, was brought about 
by my close contact with the organization. 

** At business college I, at one time, was seriously con- 
sidering Court reporting or accounting but I forgot these 
when I received my job with the engineering company. 
‘‘From the time I received this job I have known that 
engineering was my field. While employed here I was 
interested in heating, ventilating, and air conditioning 
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and plumbing engineering. Naturally, since I was work- 
ing with it day and night for 14 years. 

**Since coming to college, I have been puzzled as to 
which branch of engineering I am best suited for and I 
have even revived somewhat my early interest in account- 
ing. Heating and ventilating seem to have lost some of 
their appeal and having worked all summer under a struc- 
tural engineer, that work has begun to appeal to me.”’ 


This résumé is certainly indicative of a well-organized and 
thought out consideration of David’s vocational plans. This, 
when supplemented by his evaluation on the last page of the 
Inquiry, sums up David’s situation : 


“*T find that I rated myself highest as Engineer, Drafts- 
man, Carpenter, and Building Contractor. Others seem 
to appeal to me as far as interest is concerned but I have 
no ability or experience in these. 

‘* As far as ability to finance engineering course, I feel 
that that is assured. Also, that I can handle the technical 
part of the job. As far as health and physical require- 
ments of the job go, I can fulfill them. 

**It seems that I have had experience in the four jobs 
I rated high as well as interest and ability and also have 
reason to expect that I’ll have an opportunity for employ- 
ment when I’m through college. 

‘*My advice to myself is: Engineering is O.K. My best 
plan is to take a general course and explore as many types 
of engineering as possible in the hope of finding something 
I’m extremely interested in.’’ 


There is not much that any counselor might add to an evalu- 


ation like that. David has considered many angles of the 
problem and has worked out a very satisfactory approach. 


SUMMARY 


These brief reports illustrate various types of situations in 
which the use of the Inquiry has proved of value in reaching 
a solution in accord with the known facts of the case. Only a 
long-time evaluation program will show if the solutions sug- 
gested in these and similar cases are valid. The validity of 
these diagnoses is not assumed to be perfect; a follow-up of 
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these cases is necessary although it is beyond the scope of the 
present research. It will be a necessary adjunct of this work 
to complete a long-time evaluation program in order to judge 
the real effectiveness of this approach. 

Some of these cases (and a total of about twenty others not 
presented here) have made possible a comparison of the results 
of the Strong Vocational Interest Blank and the Occupational 
Orientation Inquiry. While not enough cases are available 
to present any statistical analyses of the results, the general 
indication is that the two blanks agree quite well in most cases. 
The Inquiry, however, has a distinct advantage in dealing with 
aspects of occupations other than interest, and this advantage 
is shown in the comparisons. In almost all cases the Inquiry 
has made available additional information which should be 
seriously considered in making any job choice but which is not 
present or indicated in the results of the Strong blank. 

Numerous cases, of which those presented here are typical, 
indicate the value of the Inquiry as a counseling technique and 
as an aid in self-evaluation. The device is practicable from 
several view-points: (1) it saves time in the interview by 
making available to the counselor prior to the interview infor- 
mation which he would normally obtain in the interview; 
(2) it requires the student to do a certain amount of the or- 
ganization of his thinking regarding his occupational plans; 
(3) it assures consideration of several important aspects of job 
choice rather than relying on one; (4) it makes full use of the 
individual’s experiences and opportunities in making the indi- 
vidual cognizant of their values; and (5) it discusses possi- 
bilities in terms of broad fields of occupations in a manner 
which facilitates clear thinking. 

However, it should be evident that these cases illustrate the 
value of the Inquiry in aiding the occupational thinking of 
students. These individuals have made their choice on logical 
bases; they seem convinced that their choices are correct, but, 
in most cases, they have alternative selections if their first 
preferences are rejected. 














INDIRECT MEASUREMENT OF LISTENERS’ 
PREFERENCES FOR MEN AND 
WOMEN COMMERCIAL 
ANNOUNCERS 


PAUL J. FAY anp WARREN C. MIDDLETON 
DePauw University 


HAT most listeners prefer men to women as radio an- 
nouncers seems to be accepted as a fact. Certainly, the 
broadcasting stations take it for granted, since the 
great majority of announcers are men. If all radio listeners 
were asked whether they preferred men or women an- 
nouneers, the majority would probably vote in favor of men. 
Yet, it seems probable that to a certain extent this expressed 
preference is stereotyped. Before the days of electrical ampli- 
fication a male speaker was less likely to strain his voice in an 
effort to be heard throughout a large auditorium. When 
women addressed large gatherings, they frequently found it 
necessary to raise their voices to such an extent that they 
sounded unnatural and forced. Some of the prejudice against 
women radio speakers, possibly, is a carry-over from experi- 
ences in listening to women platform speakers. Also, since 
most public speakers are men, the belief that men sound more 
‘‘natural’’ may be a result of the tradition that it is more 
‘‘natural’’ for men to be public speakers. Most people, we 
assume, would consider a male barber more ‘‘natural’’ than a 
female barber. 

In the present experiment an attempt was made to measure 
indirectly listeners’ preferences for men and women an- 
nouncers. By this indirect measurement, we believe, the 
effect of prejudice with respect to the preferences was reduced. 
The material read by the speakers consisted entirely of com- 
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mercials. There were two reasons for this limitation. In the 
first place, advertising announcements are peculiar to radio. 
Since there is no comparable field in platform speaking, it 
might be assumed that a dislike of women platform speakers 
would be less likely to affect listeners’ preferences. In the 
second place, the most important single function of radio an- 
nouncers (apart from routine station identification) is adver- 
tising. 

It is possible that listeners prefer men announcers for ad- 
vertisements of some types of products and women announcers 
for advertisements of other types of products. In the present 
experiment an attempt was made to test this hypothesis. 

The only previous studies of listeners’ preferences for men 
and women announcers which we could find reported in the 
psychological literature were made by Cantril and Allport 
(1, pp. 127-138). Cantril and Allport were primarily con- 
cerned with the problem of the voice characteristics which affect 
preferences for men announcers, and with the problem of the 
relationship between these preferences and the type of mate- 
rial read. Their listeners reported that the voices of the 
men speakers were more persuasive and more natural than 
the voices of the women, and that the men took more of an 
interest in the material they were reading. On the other hand, 
despite the fact that the women’s voices were reported to have 
more affectation, they were also judged to be more attractive. 
On the surface these two latter findings seem to be somewhat 
contradictory. 

The present experiment was performed in the Radio Re- 
search Laboratory of DePauw University. The fidelity of 
reception of the public address equipment employed was equal 
to that of a good radio receiver. Electrical transcriptions of 
the commercials, rather than the live voices of the announcers, 
were used in the experiment. This insured that all groups of 


1A detailed and illustrated description of the equipment and facilities 
of the DePauw Radio Research Laboratory may be found in Fay, P. J., 
and Middleton, W. C., ‘‘The DePauw Laboratory for Research on the 
Psychological Problems of Radio,’’ Amer. J. Psychol. (in press). 
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listeners heard identical voices. Also, since the speakers knew 
that their voices were not being broadcast, they spoke with 
greater naturalness and with less tension. 

The announcers were fourteen men and fourteen women 
_ college students.? All of them had had one year of college 
training in speech. In the opinion of their speech instructors 
the two groups were equal in oral reading ability. The twenty- 
eight speakers who exhibited the greatest self-confidence and 
skill in speaking into a microphone were selected from a larger 
group after a preliminary audition. None of the speakers had 
any noticeable speech defects or sectional dialect. None of 
them knew the purpose of the experiment. 

Before coming to the studio the announcers were instructed 
to practice reading their scripts and to make their reading as 
natural and as persuasive as possible. They were recorded in 
groups of four or five at a time. The speakers were seated in- 
formally about a studio table on which a non-directional micro- 
phone (RCA pressure-operated) was placed. Previous in- 
struction and practice in the proper use of the microphone 
made them sound less ‘‘read-y’’ and unnatural.® 

Forty-two different commercials, modeled after existing 
radio advertising plugs, were used as scripts.‘ They were 
equated in several respects: each advertisement began with an 

2 Cantril and Allport used only five speakers of each sex. Yet, they 
state that ‘‘the use of five speakers of each sex would, of course, tend to 
cancel out chance differences in the appeal of individual voices. . . . In a 
preliminary experiment . . . ten other speakers participated. The results 
of this experiment were substantially the same as those reported in this 
chapter.’’ (1, p. 129.) The use of fourteen speakers of each sex should 
still further insure that the ‘‘ findings are due to genuine differences in the 
appeal of the sexes and not to our sampling of speakers.’’ 

8 We have found that speakers untrained in radio sound less ‘‘read-y’’ 
and tense when they are grouped about a microphone than when they are 
alone in the studio. Instructing each one to ‘‘talk’’ his material to the 
individual seated across from him helps to reduce the artificiality char- 
acteristic of speakers when they first address a microphone. 

4 Miss Sharlee Mayer, a student in the Department of Psychology, wrote 
all the scripts and acted as general announcer in the experiment. 
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interest-arousing statement; each mentioned the brand-name 
three times; and each closed with an action statement. The 
use of nonsense dissyllables as brand-names avoided the effect 
of association with well-known brands. Each commercial had 
a reading-time of approximately twenty-five seconds. 

The commercials were divided into three groups: fourteen 
advertised products which appeal to men; fourteen, products 
which appeal to women; and fourteen, products with equal 
appeal to both sexes. The scripts in the respective groups 
were specifically addressed to men listeners, to women listen- 
ers, and to listeners in general. 

The advertisements of the following products were directed 
to men: 


Boman Shirt Filip Suit 
Taran Watch Tarot Rifle 
Baron Pipe Ganen Shaving Cream 
Mokar Tobacco Bufit Shoe Polish 
Razof Electric Shaver Zemol Hair Tonic 
Razup Shoes Darif After-Shave Lotion 
Balot Beer Malat Tobacco Pouch 
A typical commercial in this group is the following: 
Did you ever miss a shot . . . and then find your game 


half a mile away by the time you reloaded your rifle? 
That would never happen with a Tarot .22. This snappy 
little rifle will fire fifteen shots just as fast as you can pull 
the trigger. You can use a Tarot as a single-shot rifle, as 
an automatic, or as a bolt-action repeater. Man... 
you’ve got a real thrill coming with your first Tarot .22. 
See one at your favorite sporting goods store today. 


The advertisements of the following products were directed 
to women : 


Marul Nail Polish Dabit Skin Ointment 
Famel Eyewash Ronel Soap 

Tipak Shampoo Norun Hosiery 
Retip Cigarettes Zakur Furrier 

Pekot Hair Dryer Torid Lipstick 

Lelur Eyelash Curler Merit Bobbie Pins 


Zarof Mascara Lazur Face Cream 


ee res 7 
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A typical commercial in this group is the following: 


Like most women you probably admire a great many 
things about a movie actress. Didn’t you ever wish that 
you had her attractive eyes and lashes? Well, of course, 
you can’t change your eyes, but you certainly can im- 
prove them. Zarof Mascara will give you the long, thick 
lashes that men find so alluring. And you can be very 
sure that your eyes will feel cool and restful. Zarof will 
make your eyes beautiful . . . safely. Do as Hollywood 
does . . . start at once to use Zarof Mascara. 


The advertisements of the following products were not di- 

rected primarily to either sex: 

Kanon Railroad Pemak Gargle 

Kimar Portable Radio Ranol Toothpaste 

Penel Motor Oil Pamot Bus Line 

Nopar Gum Keral Cold Tablets 

Nukar Candy Bar Dural Fountain Pen 

Polar Soft Drink Zekon Car 

Pamel Cigarettes Telar Aspirin 


A typical commercial in this group is the following: 


Your doctor never takes any chances with a sore throat. 
He knows it’s too dangerous. Why don’t you do as he 
would recommend ... gargle with Pemak? It kills 
eighty per cent of those cold germs in three minutes. 
Think of that! Eighty per cent in only three minutes. 
Don’t let your cold develop into deadly pneumonia. Stop 
at your druggist’s and get an economical bottle of Pemak. 
Then, when your throat begins to have that raw feeling, 
just gargle with soothing, healing Pemak. 


Each announcer read three scripts, representing one of each 
of the three types of commercials. The advertisements of 
men’s products, of women’s products and of neutral products 
were mixed in a random order. The three orders in which each 
announcer was heard are listed in Tables 1 and 2. The same 
commercials were read by both groups of announcers. The 
scripts read by each male announcer were read by three female 
announcers, and those read by each female announcer by three 
male announcers. 
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The listeners were one hundred sixty-six students (one hun- 
dred sixteen men and fifty women) in introductory psychol- 
ogy classes.° Each listener was furnished a mimeographed 
rating scale on which were listed advertisement numbers from 
1 to 42. Each advertisement number was followed by the scale 
0, 1, 2, 3, 4. The following instructions were printed on the 
rating scale and were read by the general announcer : 


In this experiment you will hear several commercial an- 
nouncements. As soon as each advertisement is finished, 
rate it for its appeal. It is very important that your 
judgment be naive and immediate; that is, not ‘‘thought 
out,’’ but based simply upon the way the advertisement 
appeals to you. Do not let your ratings be affected by 
your preference for the products advertised. Rate the 
advertisements as such. Be sure to rate every advertise- 
ment, the numbers of which will be called by the an- 
nouncer. 


Rate each advertisement by drawing a diagonal line 
through the number indicating the degree of appeal the 
advertisement possesses. Try to let the same number 
indicate the same degree of appeal throughout the ez- 


periment. The significance of the rating numbers is as 

follows : 0—Entirely lacking in appeal; 1—Slight amount 

of appeal; 2—F air amount of appeal; 3—More than aver- 

age amount of appeal; 4—Unusually great amount of 

appeal, 

5It may be objected that college students are not ‘‘representative’’ 
radio listeners. Cantril and Allport used as listeners eighty people on 
relief (1, p. 128). There is some question as to whether reliefers or college 
students are more ‘‘representative.’’ As a matter of fact, we are rather 
skeptical of the possibility of securing a ‘‘representative’’ group of listen- 
ers in any laboratory experiment. A laboratory set-up is artificial—in 
psychology or in any other science. An investigator must choose between 
the ‘‘naturalness’’ and lack of control of a field study and the artificiality 
and scientific rigor of a laboratory study. The acquisition of a respectable 
amount of knowledge concerning the psychology of radio will, in our 
opinion, require contributions both from the field and from the laboratory. 
Incidentally, one of the present authors has made several studies of 
laborers and housewives in which he encountered considerable difficulty 
in securing satisfactory numerical ratings. It seems probable that studies 
involving the use of numerical ratings must be limited largely to high 
school or college students. 
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The listeners were not told the purpose of the experiment, or 
that any announcer was to appear more than once in the pro- 
gram. 

At the beginning of the program and after each group of 
seven commercial transcriptions recorded music was played. 
In the first experimental session the listeners rated the adver- 
tisements read by the fourteen men announcers. Eight days 
later the same listeners rated the advertisements read by the 
fourteen women announcers. 

The mean ratings of the appeal of the commercials read by 
the men announcers are given in Table 1; the mean ratings of 
the appeal of the commercials read by the women announcers 
are given in Table 2. The ratings for some speakers—irre- 
spective of the type of product advertised—are seen to be 
quite consistent. Male Announcer Le, for example, was rated 
highest in each of the three types of commercials. On the other 
hand, Male Announcer Sc was rated fourteenth in his adver- 
tisement of a man’s product, third in his advertisement of a 
woman’s product, and second in his advertisement of a neu- 
tral product. Some of this inconsistency is undoubtedly due 
to variation in the appeal of the commercials themselves. The 
variables in the experiment would have been reduced, if the 
commercials could have been previously equated for appeal. 
A comparison of the ratings of corresponding scripts® in the 
two tables, however, reveals that some commercials were more 
appealing when read by men and some more appealing when 
read by women. It seems probable that some voices were better 
adapted to certain advertisements than to others. 

The three types of commercials, as groups, appear to differ 
in appeal. Both men and women listeners rated advertise- 
ments of women’s products the lowest of the three groups, 
irrespective of the sex of the announcers. They rated adver- 
tisements of men’s products the highest of the three types read 


6 Corresponding numbers in the ‘‘Orders of announcing’’ columns of 
Tables 1 and 2 refer to identical commercials. For example, both Male 
Announcer Kn and Female Announcer Elt appeared first in their respec- 
tive programs, and both read the same advertisement of Marul Nail Polish. 














‘(991 = N) saouoqsry [ye Aq sBuryer uvoyy ; 
"(0G = N) saouoqsty uowIOM Aq sZuryeI UOT, o 
‘(OTL =N) saouoqzst] uom fq sBurze1 uve p 
*1Q0UNOUUY YORI 1OJ SHUIZVI JO UVOUT PUBID » 
*p 0} 0 wWosz opvos yurod-¢ v uo pvodde 107 yuoWMaSTJABAPS YOVE PozVI SIOU94ST] OIL, q 
‘syonpoid jeayneu 30 pue ‘syonpoid s,uowom Jo ‘syonpoid 
8, UM JO SyuoUTESTJI0Ape 04 ‘Ajoatjoodser ‘10Jo1 NN puw ‘My “PY SSurpvoy uumMoo oy], ‘prvoy SVM JOOUNOMUE YOvO YOTYA UT SIOPIO 921} OWT, v 

















103 03 = 96'T 10S 86 60S——is«0 eg Se | sis WTSSCITS ueoW 
ost sot SFT est 86 LT BHT 63 sft Set ae oe eS le: ox 
£03 40% = 86'T eet 008 I6T ITS 08886 Ss 2S ae eS Coo WW 
913 183 #13 803 «6808 = 802 est I6T 8s2'T — 2. we. Se 2 1p 
LT) = SBT BLT bLT 86 S8T(OLTT ne ss rn. = e+ s&.2e eM 
FOT 86ST O66T oT ST LT Lt Skt 8 BLT so aft 23.88 8 AL 
893 «688s L'S 993 «19S 8 193 6S «= BES CG'S n/n |. 2 2X oT 
SOT 66T 061 els 83s 10% OT 8«=OLT  OTOTT 00% 86T 08 8 8 Of ig 
S6T £03 86T 89T OLT L9T 6ts 383 86'S 661 903 96T 68 98 wt °H 
CLT (OCT 8 (SLT 99T 99T 99'T THT 9ST ET 93 868hS)6«=OOTS—iESC(‘éiéi:Ci«SK’ ve 
ost 98T 6L'T est 961 8L'T OPT BT LmT ee! Se ee Le) 
913 «1S 80% 983 393 FBS SLT 00% sot oe 2. 4... * && 77 of 
003 13 s6T ess 9s eh ) nS 5 mS 3: 2 ease ag 
403 «= L6T |B 933 sts 083 oT OST SLT ae ae: Bes oe on 
003 80% L6T oe sST 8 OLT OFT we ae: See. 3 uy 
TeyO]L wWouloMy usyy T2830 ,, Woeulo ny ie) 1 \g T®}O],, Wolo ny voy 1870] eWoTMOM pusoyy N M Ww ‘uuy 
994 180d w0* sjonpoid [erjnen sjyonpoid s, wowl0 AA sjonpoad s, uayy ee 
J0 S19piQ 


q8}uowestzl0Ape jo jvodde jo sZuryel 8, 10U94sV'T 








SLIDUNOUUP UIT YPM spuamasyssapp paguosuvsy fo joaddp ay, fo shbuyvy uvay 


T @TaAVL 

















"(991 = N) saouoqst] [Te 4q sZuryer uvopy ; 
‘(S$=N) ssoueqst, uomom Aq sZurje1 weay_ o 
‘(STLT=N) saouezst, uom Aq sZurjyer wve7_ p 
“199UNOUUB YIVE OJ SBurjel JO UvoU puvIDH o 
"p 0} Q Woz o[eos yurod-¢ wv uo jeodde 10J JUOTASTJIOAPE YOV PoyeI SAVUOZST] OLY, q 
‘sjonpoid ,eajynou yo pus ‘syonpoad s, uemom jo ‘syonpoad 
§, Wout JO SyuoUIEsTIOApV 04 ‘AjoAT}OOdser ‘1OJo1 NV pue ‘My ‘WW SZurpeoy uumNjoo oy], “psvoy SA JODUNOUUL YORs YOIYM UT SIOP1O 914} OY, « 














0s'T cst 6L'T ost 96°T 66 L 39'T 6L'T 6o°T LST cesT LST uve 
0s'T es'T 6F'T sol #'T 09°T 09°T €9°T so €e'T rae | OFT Ob 8f 7@) 
c9'T sot 89'T 9F'T 62° oe | 6e°T cet OFT Its Oleg a oo 2. eS ae 
Ts'T 98'T 8L'T TéT 00°% LST 861 90°S t6'T eo est oT oe 8. eo ee 
LL'T 98°T 9L'T 66°T ove céT eo'T 69°T OFT 0st 6L'T 08'T Se OB -2- > 
es'T 69'T SFT PLT T8'T OLT LOT 80°T 90°T 9L°T 96°T 89'T a: 2.2 Id 
Lgl r8'T LYT 68°T Lv LLT 86° 8e'T 68" cst 96°T 1st 9 oo =.” -_ 
LéT 60°3 00°% r6'T 133 sl £33 g's 61% CLT CLT cLt a ee oy 
8ST eS'T T9'T 06'T 0% 98°T 80°T 86° etl 9LT 8o'T €8°T aaa tA 
06°T T8'T r6T 6L'T cot cs'T STS £3°S oUs LL'T 9¢o°T 98°T S “.. &- -S o”g 
93°3 LTS oes 133 Ove Sas 60°S a 80% 6h'S 61'S Lg°S eae ee #d 
£03 86T 90°% 0°S 90°% £0°% cL’t 9o°T €8'T Tes Tes 1€°S 3 aw ee afte 4O 
6'T 00°% 88°T 30% 80'S 00°% LUT 88'T elt cé'T 40% 36 €t 6 ee oes TA 
9¢°T OLT Tet €8'T 00% CLT 9FT L9'T sel OFT SPT 6E°T ct 8 3g “ ng 
Lv3 82°s e's Lo°S e's 19% t9'T STS PFT Tes ces 63'S uf Rap ege Wa 
[ejO], woMOAM Udy [8}0], womOM ud, [e}Oy, womloM Udy [BO], ewOMIO A, puesy a “uuy 
oo} ts0du10g sjonpoid yerynen sjonpoid s, WaUIO AA sjonpoad s, uoyy ‘eeu 
JO S19piQ, 


q8}uomstj10Ape jo teodde jo sZurjel 8, 19u94sT'T 








SLIOUNOUUP UIWOY YPM Spuamasij.vapp paquosuvsy fo joaddp ay} fo shuoy uvay 


& ATAVL 








| 
| 
| 








COMMERCIAL ANNOUNCERS 567 


by men announcers, and advertisements of neutral products 
the highest read by women announcers. 

Our results agree with those of Cantril and Allport (1, p. 
133) in showing that women listeners rate both men and women 
speakers higher than men rate them.’ The only exception is 
with respect to women announcers reading advertisements of 
men’s products. These sex differences in rating, however, are 
small and unreliable: the critical ratio of the difference (.08) 
between the mean composite ratings of the men announcers is 
0.47 ; the critical ratio of the difference (.06) between the com- 
posite ratings of the women announcers is 0.35. Men agree 
with women listeners in their ratings of women announcers 
slightly more than they do in their ratings of men announcers. 
The coefficient of correlation between the men’s and women’s 
ratings of men announcers is .71 + .09; the corresponding r 
for women announcers is .80 + .07. 

The split-half reliability of the ratings of the individual 
commercials is extremely high. The reliability coefficient of 
the ratings of the men announcers is .95 + .004; corrected by 
the Spearman-Brown prophecy formula the r is .97. The 
corresponding reliability coefficient and corrected r of the 
ratings of the women announcers are, respectively, .97 + .002 
and .98. 

The question may be raised as to whether the technique em- 
ployed in this experiment is a valid indirect measure of pref- 
erences for men or for women announcers. Are the composite 
ratings (that is, the grand means of the ratings of the three 
advertisements read by each announcer) really measures of 
anything? As mentioned above, inspection of Tables 1 and 2 
reveals considerable consistency in the ratings for some speak- 
ers. Additional evidence of the validity of the technique is 
furnished in Table 3. The ratings of each of the three types 


7 This agreement exists despite the fact that in Cantril and Allport’s 
experiment the listeners were instructed to rate the voices for persuasive- 
ness, speaker’s interest, knowledge of personality, naturalness and attrac- 
tiveness, while in the present experiment the listeners rated the advertise- 
ments for appeal. 
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TABLE 3 


Correlation between Ratings of Each Type of Advertisement and Com- 
posite Ratings for Announcers 











Composite ratings 
All means* Other two means> 
Men Women Men Women 


announcers announcers announcers announcers 





r PE r PE r PE r PE 





Men’s products ...... 65 01 Bg | 01 mm 6: BS. 
Women’s products .58 01 73 01 34 02 22 02 
Neutral products.. .78 01 83 01 43 02 50 .02 





* Grand means of the ratings of all three types of advertisements read 
by each announcer. 

> Grand means of the ratings of the other two types of advertisements 
read by each announcer. The ratings of the advertisements of men’s 
products read by men announcers, for example, were correlated with a 
composite of the ratings of the advertisements of women’s and neutral 
products read by the same announcers; the resulting r is .24. 


of commercials for the respective announcers were correlated 
with the composite ratings of the same announcers. The an- 
nouncers, in other words, were the common elements; the ad- 
vertisements were different. For the men announcers the r’s 
range from .58 to .78; for the women announcers the r’s range 
from .71 to .83. These coefficients, of course, are spuriously 
large, because the composite ratings include the ratings of the 
particular type of commercials with which they are correlated. 
The amount of the spurious element is .57.2 When the ratings 
of the advertisements of one type of product were correlated 
with the composite ratings of the advertisements of the other 
two types of products, the coefficients were much smaller, but 
all positive. For the men announcers the r’s range from .24 
to .43, and for the women announcers from .22 to .50. In these 

8 Derived by the formult yt/s, in which ¢ equals the number of ratings 


of a particular type of commercials, and s equals the number of ratings 
in the composite. 
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coefficients there are no spurious elements. The low correla- 
tions may be attributed partly to variation in the appeal of 
the commercials, and partly to the probability that some voices 
are more effective in some advertisements than they are in 
others. 

When the ratings of the advertisements read by the men 
were correlated with the ratings of the same advertisements 
read by the women, an r of only .54 + .02 was obtained. In 
this case the advertisements were the common elements and 
the announcers were different. 

In Table 4 the mean ratings of voice appeal of men an- 
nouncers are compared with the mean ratings of women an- 
nouncers. Not only are all the differences in favor of the men 
announcers, but they are all statistically reliable. None of 
the critical ratios (difference divided by the standard error of 
the difference) are below 2.30, and in only three cases are they 
below 3.00. Both men and women listeners rate men an- 
nouncers higher for all three types of advertisements. The 
difference in favor of the men is greatest for advertisements 
of men’s products, and least for advertisements of neutral 
products. These results do not support the supposition that 
women are preferred to men as advertising announcers of 
products appealing to women. As a matter of fact, the differ- 
ence in favor of the men is greater for advertisements of 
women’s products than it is for advertisements of neutral 
products. 

There is, of course, overlapping in the distributions: some 
women are preferred to some men announcers. When we 
examine the mean ratings (by all listeners) of individual ad- 
vertisements, we find that men receive higher ratings in 
twenty-seven commercials, women receive higher ratings in 
fourteen, and in one commercial the announcers receive the 
same rating. This overlapping, however, does not lead to con- 
clusions that are particularly encouraging to women. In the 
case of the twenty-seven commercials for which men are pre- 
ferred, the mean rating for the men is 2.11 and for the women, 
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TABLE 4 


Differences between Mean Ratings of Voice Appeal of Men and Women 
Announcers for Each of Three Types of Advertised Products 





Listeners Type of Announcers M o Da Op D/op 











product 
Men’ Men 2.11 91 .24 .0283 8.48 
> Women 1.87  .96 
w a Men i977 °C 3 3; OS 6.36 
— Women 1.59 .97 
— M 2.00 99 08 0283 2.83 
en ‘ J d .02 : 
Neutral Women 1.92 .94 
c iteb Men 1.96 S86 17 O141 123.06 
Ritts Women 1.79 .97 
Men’ Men 2.14 1.04 .29 .0566 5.12 
— Women 1.85 95 
‘ene Men 1.88 98 .16 .0566 2.83 
— Women 1.72 .96 
Women 
Neutral Men 2.00 i111 18 0566 2.30 
— Women 1.96 1.00 
Cc it Men 2.04 1.05 .19 .0283 6.71 
— Women 1.85 .98 
Men’s Men 2.12 97 25 .0283 8.83 
Women 1.87 .96 
Waihien *s Men 1.83 98 .21 0.283 7.42 
Women 1.62 .97 
ry M 2.01 1.02 09 0283 
en r 7 J d 3.18 
Newton Women 1.92 95 
Composite Men a). 281. 2 C26 14.00 


Women 1.80 .97 





* Difference between mean ratings in favor of men announcers. 
> Grand mean of ratings for all three types of products. 


1.72; the difference is .39. In the case of the fourteen commer- 
cials for which women are preferred, the mean rating for the 
women is 1.95 and for the men, 1.76; the difference is .19. In 
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other words, women are preferred to men announcers only 
when the men are decidedly lacking in appeal. And even in 
these cases the superiority of the women is relatively slight. 
From a practical standpoint the cases in which women are pre- 
ferred to men can probably be ignored: broadcasting com- 
panies are unlikely to hire announcers whose voices are below 
average in appeal. 

When the speakers in Cantril and Allport’s experiment read 
fictitious advertisements,® there were no statistically reliable 
differences between the ratings of the two sexes (1, p. 135). 
Men listeners rated the female voices slightly higher in per- 
suasiveness, speaker’s interest, naturalness and attractiveness ; 
women listeners rated the male voices slightly higher in the 
same characteristics. Our results reveal no similar tendency. 
Nor, on the other hand, do they reveal any tendency (found 
by Cantril and Allport when their speakers read material 
other than fictitious advertisements) for listeners of each sex 
to rate higher speakers of the same sex. 

Any conclusions drawn from the results of this study apply 
only to commercial announcers and—possibly—only to lis- 
teners with a cultural level equal to that of college stduents. 
Further investigation is needed to determine the extent to 
which men or women are preferred for broadcasting such 
material as household advice, beauty hints, advice on child 
care, and even news commentary. It is possible, for example, 
that there is a place for women in broadcasting similar to that 
filled by ‘‘sob sisters’? on newspapers. The problem of the 
most effective types of voices for different kinds of radio speak- 
ing is an important one—much more important than the pau- 
city of experimental studies on the question might lead one 
to suspect. 

The conclusions may be summarized as follows: 

1. The effect of prejudice on listeners’ preference for men 


® Cantril and Allport used only advertisements ‘‘of standard products 
of equal interest to both sexes’’ (1, p. 130). In the present study less 
difference between the ratings of the two sexes was found for advertise- 
ments of neutral products than for the other two types of commercials. 
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as compared with women announcers probably may be re- 
duced by indirect measurement of the preferences. 

2. When the preferences are measured indirectly, both men 
and women listeners prefer men to women as commercial an- 
nouncers. This may indicate that, irrespective of any preju- 
dice on the part of listeners, men’s voices are actually more 
persuasive or convincing. 

3. Some women are preferred to some men announcers, but 
only when the men have relatively unappealing voices. 

4. Men are preferred for advertising any of the products 
commonly advertised by radio, irrespective of the sex of the 
consumers to whom the products appeal; men are even pre- 
ferred for advertisements of women’s products. 

5. There is a slight tendency for women listeners to rate all 
commercial announcers higher than men rate them. 
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THE EFFECTS OF OCCUPATIONAL MUSIC 
ON SCRAPPAGE IN THE MANUFAC- 
TURE OF RADIO TUBES’ 


JOHN F. HUMES 
Pennsylvania State College 


HE purpose of the present investigation was to study the 
effects of music upon female workers in a plant in which 
the playing of phonographic music had been an estab- 

lished practice for five years previous to this experiment. 
There was some reason to believe, because of the use of music 
as a working ‘‘condition’’ over so long a period, that any bene- 
fit to production had long ago been obtained.” Since no experi- 
ment had been conducted to confirm such a belief, or to deter- 
mine any other possible effects of music, it was suggested that 
a study be made to determine whether or not production was 
influenced by the presence of music, by its absence, by differ- 
ent kinds of music, t.e., fast or slow, or by different intervals 
of playing time. To ascertain these specific influences, then, 
was the aim of this study. 

Previous Studies.—The effects of music on workers engaged 
in simple repetitive tasks has been investigated by Wyatt and 
Langdon (1). They reported that ‘‘during the time when 
music was played the increase in output varied from 6.2 to 11.3 
per cent, while the total daily output showed an increase of 
2.6 to 6.0 per cent.’’ It is to be noted that music was experi- 


1 This study was reported in part to the American Psychological Asso- 
ciation at the meeting held in Columbus, Ohio, September 1938. An agree- 
ment between the company in which the experiment was conducted, and 
the writer has prevented an earlier publication of the results. 

2 Only normal fluctuations in production had been observed by manage- 
ment, when, during a period of several days in which a new amplifying 
system was being installed, music was not used. 
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mentally introduced by these investigators into a factory 
which previously had not made use of it. 

While there occurs considerable popular reference*® to the 
use of music in industry, surprisingly little experimentation, 
aside from the British study noted above, has been reported. 
Diserens (4) made observations of dynamometer grips, with 
and without music, but this cannot be considered an experi- 
ment conducted under purely industrial conditions. A similar 
commentary might be made on several other experiments (5, 6, 
7, 8, 9, 10) which have, nevertheless, obvious implications for 
industry. Ramsay, Rawson and others (11) have reported the 
results of a survey made in Great Britain on the prevalence 
of music in factories, the attitudes of employers to its use and 
other phases of the problem. While this was not an experi- 
mental study the investigators nevertheless concluded that 74.5 
per cent of those employers using music considered that it 
increased efficiency. 


METHOD AND PROCEDURE 


Locale.—The experiment was conducted in the plant of the 
Hygrade Sylvania Corporation, located in Emporium, Penn- 
sylvania.* This company manufactures radio-tubes, and in 
this branch from 800 to 1200 people normally are employed. 
Approximately 75 per cent of the employees are women. 

Subjects—Originally it was intended to make a series of 
preliminary observations, the results of which were to form the 
basis of a more protracted experiment. Radical operational 
changes which were introduced shortly after the preliminary 
8 Only two typical examples are cited (2, 3). 

4 The investigation was made while the writer was engaged in exten- 
sion work, conducting classes in Foreman Training, and Industrial Psy- 
chology for Executives. To Messrs. M. D. Burns, C. W. Hudson, C. B. 
Eckles, and I. E. Buck, executives who acted in the capacity of an ad- 
visory committee, and supervised the procedure on days when the experi- 
menter was not present at the piant, as well as to Mr. Dean Jordan, who 


had charge of the record-playing equipment, the writer owes a debt of 
gratitude for their hearty cooperation. 
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series had been finished made further experimentation at that 
time not feasible. However, the preliminary results were con- 
sidered important enough to report. 

This study was restricted to 88 female operators, not all of 
them doing the same kind of work, but each one was continu- 
ously employed in some of the tube assembly operations. The 
work was repetitive, requiring a high degree of hand and 
finger dexterity. Only those workers with at least one year’s 
service were included in the study, but the average length of 
service of our subjects was 6 years, 3 months. The average age 
of this group was 26 years, 3 months. 

Equipment.—A portable automatic phonograph which plays 
consecutively 8 records (10” or 12”) was used. Amplification 
to all sections of the plant was effected over a Phileo Emer- 
gency Announcing System. Approximately 400 recordings 
were always available from the plant library. 

While many different tube models were assembled during 
the course of this study, cbservations were limited to the three 
models A, R, and Z, since these had been in almost continuous 
production over a period of years. 

General Procedure.—Music had been introduced into this 
plant primarily for the purpose of demonstrating the utility 
value of the radio-tubes there produced. That the music pro- 
vided ‘‘entertainment’’ for the workers was soon recognized 
by management but no scientific effort was made to determine 
whether or not such diversion had any effect upon production. 
It was observed that the music certainly produced no detri- 
mental effects upon output, and if output increased over a 
period of time, it was not generally suspected that music had 
had anything to do with it. The playing of two to three re- 
cordings every hour of the work spell had been general prac- 
tice for five years. The selections were made by someone in 
charge of the record-playing equipment. Not much thought 
was given to the suitability of the selections, so that it might 


5 Manufactured by Philco Radio Co., Philadelphia, Pa. Description of 
apparatus may be obtained from the manufacturer. 
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be said that the daily choices were haphazard in nature. Occa- 
sionally a worker would request certain numbers to be played, 
whereas there would be, from time to time, some objection 
raised by the more sensitive individuals to selections which 
were not liked. 

For three weeks before actual experimentation was started 
the writer spent considerable time in the plant on the two days 
of the week when foreman training classes were held. While 
his presence excited some little curiosity at first, he soon be- 
came known as the ‘‘teacher,’’ and at no time during the study 
was there any evidence that the operatives suspected him of 
conducting an experiment or that there was an experiment in 
progress in which they were the subjects. 

During this time the musical selections to be played were 
chosen in the random manner heretofore described. Two selec- 
tions were played every hour of the eight-hour day; the total 
approximate playing time was 64 minutes per day. This value 
is based on numerous observations of playing time made at 
various intervals of the period. Records of 10-inch diameter 
played for approximately 3 minutes, while the 12-inch records 
played approximately 4} minutes. As the choice of the selec- 
tions was not made on the basis of record-diameter, variations 
in playing time occurred from day to day. An average of 
the observed times was computed not only for this period 
(‘‘cheeck’’) but for the subsequent periods as well. These com- 
putations are not considered sufficiently significant to present 
in this report. The term ‘‘music-hours’’ (see Table 1) has 
been introduced to indicate the total amount of time during 
a day, or a period, in which music was used. In a sense, this 
period may be considered as a ‘‘control’’ period, inasmuch as 
conditions differed in no respect from those which ordinarily 
prevailed. In order to increase the number of data to be con- 
sidered production records for two weeks preceding the period 

6 These data, as well as the original records from which the values, later 


to be represented in this paper, were computed, are on file at the Depart- 
ment of Psychology of The Pennsylvania State College. 
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just described, were collected; and this combined time (total 
of 5 weeks) is designated for the purposes of this experiment 
as the ‘‘check’’ period. 

A second period of 3 weeks is called the ‘‘slow’’ period. 
During this time, as in the periods which followed, the experi- 
menter chose the selections which were to be played and indi- 
cated the playing schedules. In the ‘‘slow’’ period, selections 
were played which were predominantly in slow tempo. 

Following this period, there was played for three weeks 
music predominantly fast in tempo. In this paper that inter- 
val is called the ‘‘fast’’ period. 

Clear-cut distinction between these two types of music is 
difficult to establish. Our categories are to be broadly inter- 
preted. Samples of the selections included in these two kinds 
of music were checked with a metronome to determine the 
number of ‘‘beats’’ per minute, and they were roughly classed 
as fast or slow.’ Those pieces which ranged from 63 to 80 beats 
per minute were placed in the ‘‘slow’’ program, while those 
which ranged between 104 to 152 beats per minute were put in 
the ‘‘fast’’ classification (12). 

A fourth period of three weeks is known as the ‘‘arranged’’ 
period. Here the experimenter attempted to place the selec- 
tions on the playing schedule in such a way that numbers quite 
familiar to the majority of subjects would occur in the earlier 
part of the programs, while the less familiar and sometimes 
more subtle selections would come in the latter part of the 
work-spell. During this period the tempos were about equally 
divided as to ‘‘slow’’ and ‘‘fast.’’ 

The fifth and last experimental period was designated a ‘‘no 
music’’ period. During the working hours no music was 
played. It was planned to extend this period three weeks, but 
on the first day of the third week the silence was abruptly 


7 Technical advice on the details of this procedure was offered by Pro- 
fessors Richard W. Grant and Hummel Fishburn of the Department of 
Music, The Pennsylvania State College. For this and other helpful sug- 
gestions pertaining to this study the writer acknowledges his gratitude. 
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broken, since a petition, signed by sixty-five workers, asking 
for the resumption of music, was presented to the management. 
Not wishing to run the risk of the possible consequences of 
refusing to grant the request, orders were issued (in the 
absence of the experimenter) to resume the programs. 

It should be pointed out that the experimental periods, as 
well as the ‘‘check’’ period, varied as to total number of hours 
worked. During the ‘‘check,’’ ‘‘slow’’ and ‘‘fast’’ periods, the 
plant was working only 3 days a week. In the ‘‘arranged”’ 
and ‘‘no music’’ periods, four days comprised the work-week. 
Moreover, occasional delays, due to temporary mechanical diffi- 
culties, resulted in fewer than 8 hours of production for one 
or another of the tube models. In a few instances unusually 
high per cent scrappage occurred due to a ‘‘run’’ of bad units 
or defective parts. Such cases, obviously not the result of 
experimental conditions, have been disregarded in the caleu- 
lations. 

Another variation in procedure involved the playing inter- 
vals. For the ‘‘slow,’’ ‘‘fast’’ and ‘‘arranged’’ periods the 
following schedule prevailed : 


First week, one selection every half-hour (approximately 
64 minutes of music daily) 

Second week, two selections consecutively every hour 
(approximately 64 minutes of music daily) 

Third week, three selections consecutively every hour 
(approximately 96 minutes of music daily) 


Table 1 shows all these data in concise form. 

For whatever light it may cast upon the interpretation of 
the results of this study, it should be noted that a ‘‘guaran- 
teed day-work rate’’ of wage payment prevailed for all oper- 
atives. Rates were generally higher than those paid by others 
in the trade. This fact, as well as favorable conditions of 
work, would seem to indicate reasonable, if not unusual, moti- 
vation. 

Individual production records were not available, hence 
group results only are considered. Because hourly production 























EFFECTS OF OCCUPATIONAL MUSIC 579 


TABLE 1 
Relative Duration of Periods and Total (approz.) Playing Time 





Total 





Days Hours a — music- 
Period Weeks per = oom — renee hours for 
week - iod 
y period (approx. ) (aa) 
Check .. 5 3 8 120 1yy 16 
Slow ........ 3 3 8 72 1} il} 
Fast ........ 3 3 8 72 1} 11} 
Arranged 3 4 8 96 1} 15 
No music 2% 4 8 68 = sav 





figures obtainable did not lend themselves readily to statistical 
treatment, scrappage was used as the production variable to 
be related to the several types of musical programs. Practi- 
cally the same degree of difficulty existed in the assembly of 
each of the tube models herein considered, but because of tech- 
nical factors the normal; or expected scrappage varied as fol- 
lows: model A, 1.4 per cent; model R, 3.5 per cent; and model 
Z, 4.5 per cent. 

Serappage was calculated for production of each of the three 
models under every one of the various conditions of the experi- 
ment in the following manner : 


Serappage was expressed hourly as a percentage of 
hourly gross output. These percentages for the total 
hours of output, under a given condition of the experi- 
ment, were averaged to obtain the mean scrappage under 
that condition, for a given model. 


The criterion of the differential effects of the various periods 
of the experiment, then, is in the statistical significance of the 
differences between these scrappage means.* 
That inaccuracies are likely to result from the averaging of 
percentages is a fact well appreciated by the writer. However, 
8 Statistical significance is here indicated in the customary manner, 


a (or more). (13). 


i.e., when ai. 
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the procedure is apparently sound in the treatment of these 
data, since all hourly raw scrap figures (number of units 
scrapped) when totalled and calculated as a percentage of all 
hourly raw production (total units produced), gave values 
practically identical with those obtained in the method used. 


RESULTS® 


Calculations of mean scrappage for each of the three differ- 
ent playing intervals—(1) one selection every half-hour, (2) 
two selections consecutively every hour, (3) three selections 
consecutively every hour—showed no consistent trends which 
might indicate that one distribution of playing time would be 


TABLE 2 
Mean Hourly Per Cent Scrappage in Every Period for Every Model 





Models Check Slow Fast Arranged No music 








Mean hourly A 2.55 1.75 2.00 2.36 2.27 
per cent R 4.89 3.30 2.85 3.64 4.70 
scrappage Z 3.43 3.48 3.78 4.39 4.94 

A 1.14 0.696 0.795 0.948 0.78 

o Distr. R 1.918 1.074 0.777 1.316 0.415 

Z 1.116 1.152 1.536 1.292 1.475 
A 0.107 0.083 0.095 0.099 0.084 
o Mean R 0.141 0.128 0.903 0.137 0.044 
Z 0.104 0.136 0.184 0.139 0.160 
NOTE 


In order to convey some idea of the magnitude of production the 
gross number of tubes (Models A, R, and Z) produced during all 
periods indicated above is shown below: 


Model A 327,290 tubes 
Model R 315,033 tubes 
Model Z 259,131 tubes 


Total 901,454 tubes 


® The writer is greatly indebted to Dr. Kinsley R. Smith of the De- 
partment of Education and Psychology, The Pennsylvania State Col- 
lege, for his guidance in the preparation of this paper, and particularly 
for his aid in the analysis of the results. 
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more favorable than another. Consequently, mean scrappage 
for the entire duration of a given period is calculated as a 
whole, without regard to playing intervals, a variable, which 
in this experiment, apparently exerted no important quantita- 
tive influence. 

In Table 2 are represented the mean hourly per cent scrap- 
page for every period for the three tube models, A, R, and Z, 
as well as the sigmas of the distributions and of the means. 

Table 3 shows the differences in per cent scrappage, the 
sigmas of these differences and the critical ratios for every 
period compared with every other period for each of the three 
models. The italicized data indicate statistical significance ; 
and the particular period in each comparison in which scrap- 
page was less is represented opposite the term ‘‘advantage.’’ 

Analysis of the data shown in the tables reveals that there 
was a higher percentage of scrappage in the ‘‘check’’ period 
as compared with that in the ‘‘slow’’ period for models A and 
R (Table 2) and that these differences are statistically signifi- 
cant is evidenced by the critical ratios 5.92 and 8.36 respec- 
tively (Table 3). Scrappage was not as great in the ‘‘check’’ 
period as in the ‘‘slow’’ period for model Z, but the difference 
is not statistically significant. 

There are differences in per cent scrappage between the 
‘*slow’’ and the ‘‘fast’’ periods. The difference for model R 
approaches statistical significance to the advantage of fast 
music, but for models A and Z statistical significance is not 
present. The ‘‘slow’’ period, compared with the ‘‘arranged’’ 
period, shows lower per cent scrappage for every model; sta- 
tistical significance obtains only for models A and Z. 

Statistical significance marks the difference between the 
‘*slow’’ period serappage and the ‘‘no music’’ scrappage for 
all models, showing the advantage of slow music. In the 
**fast’’ period scrappage is lower for models A and R when 
compared with the ‘‘check’’ period. There is in each case sta- 
tistical significance. For model Z scrappage is higher for the 
‘*fast’’ period than for the ‘‘check”’ period, but the difference 
lacks statistical significance. 
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For all three models scrappage was lower in the ‘‘fast’’ 
period as compared with the ‘‘arranged’’ period. An ap- 
proach to statistical significance is noted in the case of A and 
Z, while definite statistical significance is observed for model R. 

The apparent advantage of fast music is likewise indicated 
in the comparison with the ‘‘no music’’ period. Statistical 
significance characterizes the differences for models R and Z, 
but not for model A although the difference favors the ‘‘fast’’ 
period. 

If the serappage of every model as it occurred in the ‘‘fast’”’ 
period is compared with the scrappage of every other period, 
it will be noted that there is a lower average per cent of scrap- 
page to the advantage of fast music in 9 of the 12 comparisons. 
Five of these 9 are statistically significant. 

The ‘‘slow’’ period compared with every other period shows 
lower average per cent scrappage in 10 of the 12 comparisons; 
7 of the 10 are statistically significant. 

When the scrappage of the ‘‘no music’’ period is compared 
with that of each-of the other periods, 9 of the 12 comparisons 
show more scrapped tubes during the musicless days than 
when music was played. In 7 of these 9 instances the differ- 
ences were significant. It will be noted that; there were three 
eases of less scrappage in the ‘‘no music’’ period, although 
none of these was statistically significant. 

If the results of the ‘‘fast’’ period are compared with all 
but the “‘slow,’’ it will be noted that scrappage is lower in 8 
of the 9 cases in the ‘‘fast’’ period, significantly so in 5 of 
them. When scrappage in the ‘‘slow’’ is compared with all 
but that in the ‘‘fast,’’ there are 8 of the 9 cases which show 
to the advantage of slow, and of these 7 are significant. 

Thus it may be observed that lower per cent scrappage oc- 
curs with relative consistency in both the ‘‘slow’’ and the 
‘*fast’’ periods as compared with the other periods; and also 
that higher per cent scrappage is frequently coincident with 
the ‘‘no music’’ period, compared with all other periods. 
From such observations two broad generalizations are appar- 
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ent: (1) either slow or fast musical selections are conducive 
to lower per cent scrappage; and (2) complete absence of 
musi¢e results in a comparatively high per cent scrappage. 

This might well serve as a decisive conclusion were it not for 
the fact that when the results of the ‘‘check’’ and the ‘‘ar- 
ranged’’ periods are further regarded, considerable doubt is 
east upon the effect of music, and the effect of its absence as 
reflected in variations in scrappage in this experiment.’® In 
comparing these two periods it is noted that per cent scrap- 
page is lower in the ‘‘arranged’’ period for models A and R, 
and higher for model Z. Statistical significance is indicated 
only for models R and Z. 

Comparing the ‘‘no music’’ period with the ‘‘check,’’ statis- 
tical significance marks the difference in per cent scrappage 
for model Z only, to the advantage of the ‘‘check’’ period. 
While the differences for the other two models favor ‘‘no 
music,’’ statistical significance is lacking. "When the ‘‘ar- 
ranged’’ period is compared with the ‘‘no music’’ period, the 
former has the advantage for models R and Z, while the differ- 
ence is in favor of ‘‘no music’’ for model A. There is statis- 
tical significance only in the case of R. 

To summarize the findings in the ‘‘check’’ and the ‘“‘ar- 
ranged’’ periods, per cent scrappage was higher in the 
*‘check’’ in 8 of the 12 comparisons, 5 cases having statistical 
significance. In the four instances in which scrappage per 
cent was lower during the ‘‘check’’ period, there is statistical 
significance in two cases. Somewhat similar results are ob- 
served in the ‘‘arranged’’ period. Out of 12 comparisons there 
oceur 8 cases of higher per cent scrappage in this period ; four 
of these carry statistical significance. Again, the four cases of 
lower per cent scrappage favoring arranged programs show 
statistical significance in two of them. 

10 It should be recalled that these two periods differed from each other 
only slightly in nature. In the ‘‘check’’ period selections were made 
haphazardly, while in the ‘‘arranged’’ period, some system was used in the 


programming but there was approximately an equal number of beth fast 
and slow pieces played in each. 


EFFECTS OF OCCUPATIONAL MUSIC 585 


DISCUSSION AND CONCLUSION 


The results of this experiment warrant only a most cautious 
interpretation. While there are certain statistical indications 
of advantage in the use of music in general, and of all fast or 
all slow music in particular, the experimental observations 
were not recorded over a sufficiently long period to justify 
definite conclusions. That both fast and slow music should be 
accompanied generally by less spoiled work than in the other 
conditions of the experiment is difficult of explanation in terms 
other than errors of sampling. It is likely that in the selec- 
tions chosen for playing, the difference between the fast and 
the slow categories was not sharply enough drawn. Indeed, 
nothing like the conventional dirge was played. To add 
further to the difficulty of interpretation of these anomalous 
results, preference was expressed by the operatives for ‘‘ar- 
ranged’’ programs made up of both fast and slow music, over 
all fast or all slow music."* It is probable that apparent incon- 
sistencies in the results were due to variables not observed by 
the experimenter, and of course not controlled. Variations in 
humidity, for example, were believed by plant engineers to 
influence scrappage. 

Whether or not the statistical findings of this study accu- 
rately represent the effects of music on production and scrap- 
page, empirical evidence based upon interviews with company 
executives and operatives, as well as the results of the question- 
naire study already alluded to, indicates that the use of music 
in this plant is not only desirable but also beneficial. 

To observe that much experimentation remains to be done 
in the field of occupational music may be trite, but it is none 
the less true. 

SUMMARY 


Statistical computations indicate that scrappage was in gen- 

11 A questionnaire study was made at the close of the present experi- 
ment, the results of which are later to be reported in detail. Tentative 
findings indicate a strong preference for contemporary dance music, with 
an incidence of from 2 to 4 selections at half-hour intervals. 
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eral higher in the absence of occupational music than during 
its presence. Scrappage was generally less during the playing 
of (1) all fast music, and (2) all slow music as compared with 
the other conditions of the experiment (‘‘check,’’ “‘arranged’”’ 
and ‘‘no music’’ periods). 

Inconsistencies in the statistical results indicate that factors 
other than music, although none was observed, operated to pro- 
duce some and possibly all of the differentiating effects noted. 

Employee morale is favorably influenced by daily musical 
programs and adversely influenced by their (programs) ab- 
sence, once music has become an established condition of work. 
This was evidenced by the petition presented to management 
by the operatives during the ‘‘no music’’ period. 

Executives of the company are convinced that occupational 
musie as used in this plant is desirable and beneficial. 
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SAMPLING ERRORS INVOLVED IN INCOM- 
PLETE RETURNS TO MAIL QUESTION 
NAIRES 


FRANK K. SHUTTLEWORTH 
College of the City of New York 


N spite of the fact that questionnaires, distributed by mail, 
| are widely used in the collection of original data, serious 
attempts to determine the margin of error in their neces- 
sarily incomplete returns are very rare. The rarity of serious 
studies is due, of course, to the extreme difficulty of obtaining 
a complete one hundred per cent return in any given practical 
situation. Most investigators who have given attention to the 
problem of sampling errors in incomplete returns to mail ques- 
tionnaires have been driven to some indirect appraisal. Thus, 
many studies compare early and later returns. If early re- 
turns amounting, for example, to 20 per cent of the canvassed 
population and later returns amounting to another 20 per cent 
are alike, it is usually assumed that the obtained 40 per cent 
return is representative. Similarly, the writer has compared 
the returns from a population 51 per cent of whom replied with 
the returns from a population only 19 per cent of whom 
replied.* 

However, it should be abundantly clear that the only ade- 
quate test of the representativeness of incomplete returns is to 
check them against complete returns. Doubtless many have 
tried to do just this, but as far as the writer is aware the only 
published reports of success in this attempt are by Toops.” ° 


1 Shuttleworth, Frank K., ‘‘A Study of Questionnaire Technique,’’ J. 
Educ. Psychol., 1931, 22, 652-658. 


2 Toops, Herbert A., ‘‘ Validating the Questionnaire Method,’’ J. Per- 
sonnel Res., 1923, 2, 153-169. 

8 Toops, Herbert A., ‘‘Returns from Follow-Up Letters to Question- 
naires,’’ J, Appl. Psychol., 1926, 10, 92-101. 
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Toops found insignificant differences between early incomplete 
and final complete returns, but was wisely cautious in his in- 
terpretation of this finding. In our own study, which also 
involves a complete return, we have found, on the contrary, a 
significant difference between early incomplete and final com- 
plete returns. 

The incentive for the present study arose in connection with 
a survey of the employment status of 327 technology and chem- 
istry majors who graduated from the College of the City of 
New York in 1936 and for whom addresses were available in 
the Office of the Alumni Association. Questionnaires, each ac- 
companied by a stamped return envelope, requiring no signa- 
ture, but identified by code numbers, were distributed by mail 
to these alumni in June 1939. Early returns from 184 of these 
alumni received during the summer of 1939 showed only half 
of one per cent unemployed, an almost unbelievably small pro- 
portion. The natural suspicion was that the more successful 
alumni had responded giving a distorted picture of the situ- 
ation. Since most of these alumni or at least their parents 
live in the New York City area, permitting home visits where 
the mail failed, it was decided to attempt the collection of a 
complete return. Accordingly, a second questionnaire dupli- 
cating the conditions of the first but enclosing a letter explain- 
ing our suspicions, was sent out in September. This was fol- 
lowed shortly by a postal card reminder. An additional 76 
replies to October 31st showed 6.6 per cent unemployed. Dur- 
ing November and December undergraduate students in tech- 
nology and chemistry were organized to contact the alumni 
who still failed to reply. Each student was given one or more 
addresses of alumni living in his neighborhood. Although 
some thirty students actively cooperated, the returns were dis- 
appointingly small partly because we attempted to maintain 
the confidential status of the inquiry and partly because of the 
large number of addresses which proved insufficient. Each 
such insufficient address initiated special efforts, particularly 
by the Alumni Association, to obtain a new and better address 
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and each such new address involved a renewed series of at- 
tempts to obtain information. When the work of student 
volunteers petered out late in January, 1940, we distributed 
the questionnaire a third time, again enclosing a stamped 
return envelope. Again the returns were meager. In des- 
peration we used deliver-to-addressee-only, return-receipt-re- 
quested, registered mail. These were much more successful in 
bringing in replies and were repeated a second and third time, 
always enclosing a stamped return envelope. Finally, the 
writer undertook a systematical personal canvas of those who 
still failed to reply. These efforts brought a complete return 
from all those for whom sufficient addresses could be obtained. 
At the end, there remained only 18 alumni with insufficient 
addresses. The insufficiency of these addresses was verified in 
each case by personal visits to the supposed address or ad- 
dresses or by the return of undeliverable registered mail. The 
final 49 replies, instead of continuing the trend toward still 
more unemployment, showed only 4.0 per cent unemployed. 
Although an exact count cannot be made, we estimate that two- 
thirds of this final group represent individuals whose original 
addresses were insufficient. 

The first 184 returns from the 327 alumni showed only .5 per 
cent unemployed, while the later 125 returns showed 5.6 per 
cent unemployed. The exact significance of the difference be- 
tween these percentages involves a neat statistical problem. 
According to the usual formula, the difference is only 2.2 times 
its standard error. The usual formula assumes, of course, that 
the two samples of 184 and 125 cases were drawn at random 
from an infinite universe. Actually, however, both samples 
were taken from a single strictly finite population totaling at 
the most 327 cases. In this situation the formula of Splawa- 
Neyman‘* would seem to be more appropriate making the ob- 
served difference many times as large as its standard error. 

In summary, it is clear, in this particular questionnaire situ- 


4 Splawa-Neyman, J., ‘‘Contributions to the Theory of Small Samples 
Drawn from a Finite Population,’’ Biometrika, 1925, 7, 472-479. 
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ation with all its special local factors, that early incomplete 
returns from nearly sixty per cent of the canvassed population 
involved serious sampling errors. The early incomplete re- 
turns tended to come from the more successful alumni. It 
would be foolish to generalize from this experience concerning 
other questionnaire situations. We may conclude, however, 
that each questionnaire situation needs intensive study. Such 
an intensive study should include a complete return from at 
least a portion of the total population. Given this experience, 
the main canvass or future questionnaire distributions of the 
same character can be planned to eliminate or minimize sam- 
pling errors. 








A NOTE ON INTERPOLATION OF KENT ORAL 
EMERGENCY TEST SCORES INTO MENTAL 
AGE YEARS AND MONTHS* 


WILLIAM SLOAN 4anp AGNES A. SHARP 
Psychiatric Institute of the Municipal Court of Chicago 


LORENCE M. TEAGARDEN presented ‘‘A Method of 
F Interpolating Kent Oral Emergency Test Scores into 
Mental Age Years and Months,”’ in the August, 1937 
issue of the JoURNAL or APPLIED PsycHoLogy. Her method 
seems to us to be seriously limited. First, each point is not 
given the same value in mental age months within a given year. 
For instance, at the eight year level, in the Middle Scale, the 
successive points have values of one month, four months, six 
months, seven months, and ten months. It is evident that one 
point in score may be equivalent to from one month to three 
months. This illustration is repeated several times in her 
scale. Second, the initial score at each year level is assigned 
a value of one month beyond the given year. For example, the 
eight-year level, on the Middle scale, falls at sixteen points 
scored. Teagarden gives score sixteen a value of eight years, 
one month. This method of interpolation holds throughout 
the table except at nine years on the Middle Scale. 

Sometime ago, after using the Kent Oral Emergency Test 
for over one year, we adopted, for practical purposes, a method 
of interpolation which is presented in Table 1. 

The essence of this method is the equal division of the twelve 
mental age months in each year. Thus, if there are three 
points for any year, each point has a value of four mental age 
months ; for five points in any year, each point has a value of 


* The authors are indebted to David B. Rotman, M.D., Director, for 
many valuable suggestions. 
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TABLE 1 


Interpolation of Kent Oral Emergency Scores into Mental Age 
Years and Months 








Lower scale Middle scale U r seale 
Score wA* 1Qt MA" 1Qt MA Lgt Sore 

5 60 31.2 5 
6 63 32.8 6 
7 66 34.4 7 
~ 69 36 8 
9 72 37.5 9 

‘ 10 75 39 10 
Y ll 78 40.6 11 
12 81 42.2 12 
13 84 43.7 13 
14 86.4 45 14 
15 88.8 46.3 15 
16 91.2 47.5 96 50 16 
17 93.6 48.7 98.4 51 108 56.2 17 
18 96 50 100.8 52.5 110.4 57.5 18 
19 98.4 51 103.2 54 112.8 58.7 19 
20 100.8 52 105.6 55 115.2 60 20 
21 103.2 54 108 56 1176 61.2 21 
22 105.6 55 110 57.5 120 62.5 22 
23 108 56.2 112 58.5 122.4 63.8 23 
24 110.4 57.5 114 59.5 124.8 65 24 
25 112.8 58.4 116 60.5 127.2 66.3 25 
26 115.2 60 118 61.5 129.6 67.5 26 
27 1176 ~3=6 61 120 62.5 132 68.6 27 
28 120 62.5 122.4 64 134.4 70 28 
29 123 64 1248 65 136.8 71.1 29 
30 126 65.5 127.2 66 139.2 72.5 30 
31 129 67 129.6 67.5 141.6 73.7 31 
32 132 68.7 132 68.6 144 75 32 
33 135 70.4 147 76.5 33 
34 138 72 150 78 34 
35 141 73.5 153 79.6 35 
36 144 75 156 81.2 36 
37 148 77 158.4 82.5 37 
38 152 79 160.8 83.7 38 
39 156 81.2 163.2 85 39 
40 160 83.4 165.6 86.1 40 
41 174 85.5 168 87.5 41 
42 168 87.5 170.4 88.9 42 
43 170.4 = =89 172.8 90 43 
44 172.8 90 175.2 91.4 44 
45 175.2 91.4 177.6 92.5 45 
46 177.6 92.6 180 94 plus 46 
47 180 plus 94 47 








* This is mental age months. 
t This is the intelligence quotient for adults only based on a 16 yr. C.A. 
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two and four-tenths mental age months; etc. By dividing the 
year equally the first point at each year level falls exactly on 
that year. 

In the Psychiatric Institute of the Municipal Court of Chi- 
cago our population is limited to adults. For this reason we 
have added a column of Intelligence Quotients for adults. The 
chronological age of sixteen years is used as a constant divisor. 
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NOTE ON A TEST OF SOCIAL COMPETENCE 


HILE conducting an investigation aimed at comparing the social 
competence of college students with that of employed and other 
groups, the writer encountered great need for a readily adminis- 

tered test of social growth applicable to persons sixteen through twenty- 
four years of age.1 This strongly-felt lack has also been indicated by 
many who deal with problems of guidance and selection, a fact not at 
all surprising when one considers that in many instances the non-intel- 
lectual, more purely social aspects of personality are of an importance 
equal to, if not greater than, sheer mental ability. In order to carry out 
the investigation referred to, the writer began the development of a 
measure of social maturity which has finally been issued as the Inventory 
of Social Behavior.2 It is the purpose of this (note) to indicate briefly 
its nature and applicability. 

The Inventory, which is self-administering and requires approximately 
fifteen minutes of the subject’s time, was originally administered to the 
following 899 subjects: 517 college students at the University of Nebraska, 
the Municipal University of Omaha, Creighton University, and the Peru 
(Nebraska) State Teachers College; 129 Civilian Conservation Corps 
enrollees at the Weeping Water, Nebraska, camp; 101 unemployed sub- 
jects obtained through the Nebraska State Employment Service; and 152 
employed persons working at various kinds of private full-time employ- 
ment in Omaha and Lincoln, Nebraska. 

The above subjects were separated into three approximately equal groups 
according to chronological age. The proportion of each group responding 
to each choice of the thirty-three items (of the multiple-choice type) was 
then determined. Validation of the items was made upon the basis of 
change with increasing age, the underlying assumption being that the 
differences which were found to exist between the youngest and oldest 
thirds of subjects occurred in the direction of increasing social maturity. 
Weights were then assigned according to the differing proportion of re- 
sponses of the youngest age group (16 through 18 years) and the oldest 

1 Weitzman, E., ‘‘Test of Social Maturity and Its Use in Comparing 
People Sixteen through Twenty-Four Years of Age,’’ in Abstracts of 
Doctoral Dissertations. University of Nebraska, Lincoln, 1940, 94-100. 

2 Weitzman, E., Inventory of Social Behavior. Test and Manual pub- 
lished by Sheridan Supply Co., Beverley Hills, California. 

3 Some of the items of the Inventory have been adapted from E. A. 
Doll’s Vineland Social Maturity Scale, with his knowledge and approval. 


595 


596 ELLIS WEITZMAN 


(21 through 24 years), using Kelley’s 1934 formula. Since there is no 
justification for setting up any one of the four groups investigated as the 
desirable standard of social maturity, the weighting was based on a com- 
bination of all groups, percentages being employed to equalize the im- 
portance of each kind of subject. 

Test-retest reliability was estimated, with an intervening interval of 
one week, with a group of 42 students at George Washington University. 
The reliability coefficient which resulted was .88, with a PE of .02. The 
standard error of measurement was 2.34, which is 28.5 per cent of the 
standard deviation of this limited group. 

As an external check upon the assumption made in validation—that 
increasing score accompanies increasing chronological age—the test was 
administered to a group of 100 subjects made up of college students at 
St. Louis University and of employed persons in Washington, D. C. 
The linear correlation of the scores with chronological age was .55, with a 
PE of .047. The correlation ratio for the regression of age-on-scores was 
54, and for the regression scores-on-age, .61. Because there is naturally 
considerable spread of social maturity at any age level, higher correlations 
with chronological age should not be expected. 

As further evidence of validity, the middle third of the originally tested 
group of 899 (19 and 20 years of age), which ages had not been used in 
determining item weights, was examined for total scores and compari- 
sons made between the four kinds of subjects of the same chronological 
ages. Group comparisons yielded the following data: 














Mess Critical ratios 
Kind of group Mean S.D. _ social Em- ay 
age ployed College CCC 
Employed .............. 46.3 7.2 Se ss 4? le 
Cole ye oo .ceccocoene 44.3 6.3 20.0 Si - && 
Lt) SR ae 42.7 5.4 19.2 2.6 er a.. 
Unemployed 39.4 5.7 17.7 4.6 4.7 2.8 





Norms based on all 899 subjects, including 409 men and 490 women, are 
presented in the Manual in terms of average scores characteristic of each 
chronological age, in terms of centiles, and for college students in terms 
of average scores for years in school. 

ELLIs WEITZMAN 
Washington, D. C. 

4 Kelley, T. L., ‘‘The Scoring of Alternative Responses with Reference 

to Some Criterion,’’ J. Educ. Psychol., 1934, 25, 504-510. 
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NEWS AND NOTES 


The Foster Parents’ Plan for War Children is operating twenty-five 
children’s colonies in Great Britain today and is caring for more than 
4,000 refugee children. Miss Anna Freud, daughter of the late Sigmund 
Freud, who directs three of the Plan’s Nursery centres, together with 
Mrs. Dorothy T. Burlington, American psychologist, has prepared several 
very interesting reports on problems of the war children of England. The 
chairman of the British Committee is J. B. Priestley, B.B.C. commentator, 
novelist, playwright and journalist. Mrs. Edna Blue heads the Ameri- 
can Committee. Children can be ‘‘ adopted’’ through the Plan for $15.00 
per month. Case histories, photographs and correspondence are sent to 
foster parents. When children are too young to write, the directors of 
the colonies send monthly reports on the progress of the children to their 
foster parents. A partial list of American sponsors and foster parents 
include Mrs, Eleanor Roosevelt, Dr. John Haynes Holmes, Mr. and Mrs. 
H. V. Kaltenborn, Helen Keller and many other prominent persons. Those 
who may wish to learn more about the Foster Parents’ Plan may write to 


the American Headquarters, 55 West 42nd Street, New York City, for 
information. 


The American College Personnel Association will hold its 1942 meetings 
at the Fairmont Hotel, San Francisco, California, on February 18 to 20. 
Other organizations in the Council of Guidance and Personnel Associa- 
tions will also make headquarters at the Fairmont. Further information 
may be obtained from Dr. D. D. Feder, Secretary, Personnel Bureau, Uni- 
versity of Illinois, Urbana, Illinois. 


Dr. Kirtley F. Mather of Harvard University, President of the Institute 
for Propaganda Analysis, has announced the election to the Institute’s 
Board of Directors of Dr. Peter H. Odegard and Dr. Maurice Rea Davie, 
also the appointment to the Institute staff of Dr. Barrington Moore, Jr., 
as a Research Assistant, and of Miss Eleanor Flexner as assistant to the 
Executive Director. 

Mr. Clyde Beals, Institute Editor, has outlined the schedule of Propa- 
ganda Analysis Bulletins as follows: June, ‘‘ American Common Sense’’ ; 
July, ‘‘Axis Voices among the Foreign Born’’; August, ‘‘The Negro 
Looks at Democracy’’; September, ‘‘ Your Health and Propaganda’’; 
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October, ‘‘ Youth Propagandists’’; and November, ‘‘The Women Talk it 
Over.’’ The last three are tentative titles. 

The Decide for Yourself Packets of study materials for the same period 
are as follows: June, ‘‘Critical Thinking in a Crisis’’; July, ‘‘The 
Loyalties of the Foreign Born’’; August, ‘‘Our Negroes and Their Prob- 
lem’’; September, ‘‘Solutions for Your Health Problem’’; October, 
**Youth Look to the Past and the Future’’; and November, ‘‘ What 
Women’s Organizations Are Saying.’’ The Institute’s office is at 211 
Fourth Avenue, New York, N. Y., where further information may be 
obtained concerning membership and subscription rates. 


The American Management Association held a conference on October 22 
and 23 at the Hotel Pennsylvania, New York City. Discussions centered 
around ways of surmounting the obstacles caused by the defense program 
and the personnel issues that are assuming new importance under today’s 
conditions. Among the general subjects under discussion in the various 
sessions included ‘‘Office Problems in a War Economy,’’ ‘‘ Emergency 
Salary Administration,’’ ‘‘ Maintaining the Office Force in the Present 
Emergency,’’ ‘‘Stepping up Office Production,’’ and ‘‘Coping with To- 
day’s Office Personnel Problems.’’ During the Thursday afternoon ses- 
sion Dr. Marion A. Bills, of the Aetna Life Insurance Company of Hart- 
ford, Conn., spoke on ‘‘ Ability and Aptitude Tests for Clerical Workers.’’ 


The New York Times published recently an article by Bess Goodykoontz, 
Assistant Commissioner of Education, U. 8. Office of Education, entitled 
**Latin America Center of Study,’’ in which she reviews efforts on the 
parts of a number of institutions to aid the relations between the United 
States and the Latin American countries. Responses to an inquiry on this 
subject sent by the United States Commissioner of Education to more than 
700 summer school directors hive been analyzed. This past summer there 
were institutes on inter-American affairs, Latin-American workshops, 
special seminars organized for study both in Latin America and in this 
country, lecture series and many additional courses in history, language, 
literature, geography, education, economics and political science. State 
universities in Alabama, Colorado, Pennsylvania, Texas and Wisconsin 
and Butler University, University of Chicago and Harvard reported the 
study of the Portuguese language. At Mills College an Institute of In- 
ternational Relations was held from June 22 to July 2. Mills also con- 
ducted a workshop in Hispanic American culture and international rela- 
tions. As a result of these various activities teachers and students in the 
United States are moving toward a better understanding of the language, 
literature, history and problems of the other American republics. 
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Federal, state, and local governmental personnel workers will gather 
at the Hotel Roosevelt in Jacksonville, Florida, on October 27, 28, 29 and 
30 for the thirty-third annual meeting of the Civil Service Assembly of 
the United States and Canada. The chief topic of interest during the 
four-day meeting will be the effect of national defense activities upon the 
whole field of public personnel administration. During the past year new 
problems related to the shortage of labor and mobilization of personnel 
resources for military defense have come to the fore. At the same time, 
the number and size of civil service agencies have expanded rapidly due to 
the rising tide of the merit system movement and the increased pressures 
for the efficient handling of expanded governmental activities. It is 
expected that the 1941 meeting will be of major importance to all those 
who are concerned with the improvement of the standards and practices 
of merit system administration. 


The Civil Service Commission of the City of Detroit, due to improved 
economic conditions coupled with National Defense Program, has found 
it necessary to expand considerably its technical personnel staff. The 
Commission has waived its residence rule for the following examinations: 
Jr. Personnel Examiner in Business Administration, in Engineering, in 
Medical Science, in Skilled Trades and Sr. Personnel Examiner in Engi- 
neering, and hopes to fill these positions with persons who have had con- 
siderable occupational experience as well as some public personnel experi- 
ence. All positions in the City of Detroit are afforded protection by a 
civil service system established more than twenty-eight years ago; pro- 
motions on the basis of merit; protection of tenure of persons on military 
leave, whether on eligible list or actually employed, and an actuarially 
sound pension plan. Applications can be obtained by writing to the 
Detroit Civil Service Commission, 15th Floor Water Board Building, 735 
Randolph Street, Detroit, Michigan. 


A new book by Ethelyn L. Stinson entitled ‘‘How to Teach Children 
Music’’ has just been published by Harper & Brothers. Miss Stinson 
states that every child—the slow, the backward, the problem child, the 
child with reading and speech difficulties—regardless of age or talent can 
learn to understand and enjoy music. Her statement is based upon the 
result of years of study and experimentation on the best and simplest 
ways of teaching music and music appreciation. She explains how to 
approach the child on the instinctive level, how to plan a course with spe- 
cial consideration for visual and auditory learning, how and what music 
to use to get maximum response, how to introduce the study of instruments, 
rhythm work, musical interpretation, composition, symphonic music, operas 
and other musical forms. 
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Correlation of the material for this book is a part of the activities of 
the Child Research Clinie of The Woods Schools, of Langhorne, Pennsyl- 
vania. The techniques set forth in the book should prove of value to all 
teachers in any school, public or private. 


The Tenth Educational Conference will be held on October 30 and 31 
at the Hotel Biltmore in New York City under the joint auspices of the 
Educational Reeords Bureau, The Cooperative Test Service, The Commit- 
tee on Measurement and Guidance of the American Council on Education, 
and the Commission on the Relation of School and College of the Pro- 
gressive Education Association. The Conference expected to attract a 
larger group than usual this year because of the outstanding speakers to 
appear on the program and the importance of their topics. On Thursday 
evening Mrs. Eleanor Roosevelt will be the guest speaker at the dinner 
session. 


The Psychological Corporation held its annual luncheon at Northwestern 
University on September 3, 1941, in conjunction with the annual meeting 
of the American Psychological Association and the American Association 
for Applied Psychology. As in former years, Dr. Miles presided and made 
brief explanation of the work earried on by the Corporation. Following 
this members of the staffs of the New York and Chicago offices were intro- 
duced, among them being Dr. Rose G. Anderson, George K. Bennett, 
Albert D. Frieberg, Arthur W. Kornhauser, and Wallace H. Wulfeck. Dr. 
Walter Dill Seott, President Emeritus of Northwestern University, wel- 
comed those in attendance to the Northwestern campus. 

Officers of the Corporation are: Walter R. Miles, President; Paul 8. 
Achilles, Vice President and Manager; Henry C. Link, Vice President and 
Treasurer; Albert D. Freiberg, Secretary; George K. Bennett, Assistant 
Secretary; and Philip G. Corby, Assistant Treasurer. 
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Ross, C.C. Measurement in Today’s Schools. New York, Prentice-Hall, 
Inec., 1941. Pp. xxiii-597. $3.25. 

The author states in his preface that, despite progress made in measure- 
ment in the last quarter century, textbooks have remained much the same. 
In educational measurement they have been largely concerned with tests of 
subject matter and have tended to early obsolescence because of the pub- 
lication of other tests. He has sought a functional approach through 
emphasizing problems relating to the intelligent use and interpretation of 
the tools of measurement rather than the tools themselves. 

The introductory part of the book contains a chapter on the historical 
development of measurement in education, and one on the characteristics 
of a satisfactory measuring instrument. Part II is devoted to the con- 
struction of informal teacher-made tests. Part III has five chapters deal- 
ing with various aspects of the testing program. Part IV has two chapters 
on motivation in relation to measurement and a chapter each on practice, 
diagnosis and school marks. .The final division is on measurement in school 
administration. 

In terms of practical value to the classroom teacher and the school 
administrator, the writer’s approach would seem to be a fruitful one. 
There is always a danger that the course in educational measurements 
may fail to take sufficient account of the simple fact that teachers for the 
most part make their own examinations and tests. The three chapters 
on teacher-made tests should be of great help to teachers in preparing tests 
that better meet the purposes for which such tests are designed. The 
objection may be raised that the book fails to give the student a sufficient 
familiarity with current standardized achievement tests. This may be 
answered in two ways. First, the writer does not avoid reference to such 
tests. Indeed, many tests are mentioned, although detailed descriptions 
with sample pages are left out as not in keeping with the writer’s purpose. 
Second, and more important, one does not teach or learn measurement from 
the textbook alone. A textbook dealing with principles, methods, and 
results can readily be supplemented by the teacher with plenty of actual 
test materials. The book is thought-provoking and covers a wide range of 
problems. It is about as interestingly written as such a book very well 
could be. On the other hand, the critical reader will probably find himself 
unable to agree with the writer at many points. He will challenge cer- 
tain statements or points of view presented in the book, or find his own 
challenged. 
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Measurement in Today’s Schools may be characterized as a sane, 
thoughtful piece of work, dealing with the fundamentals of educational 
measurement. Not only will it be valuable as a textbook for the teacher 
in training but it merits a wide reading by teachers, administrators and 
those laymen who as non-professional students of education, endeavor to 
keep themselves informed on current educational practices. 

Amos C. ANDERSON, 
Ohio University 


PINTNER, Rupo.r, EIsENSON, JON, and STANTON, MILDRED. The Psychol- 
ogy of the Physically Handicapped. New York, F. 8. Crofts & Co., 
1941. Pp. 391. $3.00. 

Perhaps the simplest thing which is to be expected of the textbook writer 
is the accuracy of the title under which his book is presented to the public. 
In the present instance the title promises too much. Actually the con- 
tents are limited to psychometric findings with the physically handicapped, 
and in some respects even these are not complete. While it is true that the 
first four general chapters on personality development, mental hygiene, the 
nervous and endocrine systems and psychological tests constitute one- 
quarter of the text, they are so elementary and there is so complete a lack 
of integration with the rest of the book that they can conveniently be dis- 
regarded. 

Chapers five to twelve are devoted to specific types of physical handi- 
caps. The most complete chapters are those on the deaf and hard of hear- 
ing; the materials on the blind and partially sighted are adequate; the 
chapter on the crippled and that on ‘‘other physically handicapped’’ are 
limited; finally, the chapters on speech defectives are of questionable 
adequacy. The organization of each chapter follows a general plan of pre- 
senting summaries of published results on psychological and educational 
measurements. These summaries give the reader an impression of a series 
of abstracts rather than an integrated resume of research findings. 

This text will probably find some usefulness as a very elementary text 
book in courses where psychological interest is limited to psychometrics. 
In no sense can it be considered a treatise on ‘‘the psychology of the 
physically handicapped.’’ 

C. M. Lourtrr, 
Indiana Uniwersity 


SHERRINGTON, Sik CHARLES. Man on His Nature. New York, The Mac- 
millan Company, 1941. Pp. 413. 

To the currently popular concept of ‘‘frame of reference,’’ there is an 

important and too much neglected negative corollary,—that our training 

and interests and purposes determine what we don’t see as well as what 
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we do see. The psychologist, in his struggle for understanding of man, 
must often suspect that his own background hides from him ideas which 
he urgently needs to know, ideas which may be common currency in the 
work of others who are thinking about man from different frames of 
reference. If this thesis is sound, the sociologists, the economists, the 
professional philosophers, the physiologists should have much rich stuff to 
offer psychologists. In this book, Sherrington attacks the problem of 
the nature of mind from the frame of reference of the physiologist, and 
one finds here both richness and dross. 

Interestingly enough, Sherrington himself feels the need for with- 
drawing from the insistent push and bustle of his own immediate 
problems in the field of physiology. He achieves this detachment, how- 
ever, not by examining physiology from another contemporary stream of 
thought but rather from a vantage point removed in time. Modern 
thought he contrasts with the view of man held by Jean Fernel, an 
eminent 16th century physician, whose views are startlingly similar to 
those of the man on the street, the student in the psychology classroom, 
today. Thus by a neat bit of indirection, the author quite disarms the 
reader. It is not the reader’s antique superstitions that fall flimsily 
before a simple telling of the story of modern biology, chemistry, and 
physics, but the once pioneer views of an ancient and honored physician. 
The reader is saved an effort at defense; he identifies himself with the 
views of the author rather than strenuously fighting off new ideas. The 
nature of life, of death, of the soul, of the influence of heavenly bodies on 
human destinies, as anciently conceived by the scholar and modernly by 
many educated people, are discussed and then contegated with the scien- 
tist’s less anthropocentrice views of man. 

And in this discussion of modern biology and physiology, Sherrington 
comes to his best. The reviewer has never read a more fascinating ac- 
count of the basic concepts of biological thinking—of microscopic life 
(‘‘life im little’’), of the nature of cells (‘‘ factories of a furious chem- 
istry’’), of ontological development. Good, too, is his discussion of 
evolution. Reading these chapters is high adventure. They are earnestly 
recommended as rich sources of information, particularly for the student 
in psychology who needs an interesting and scholarly review of some 
of the basic ideas in biology. 

One suspects that Sherrington might willingly have stopped here. 
But the Gifford Lectureship urges its holders into fields where skill as a 
specialized scientist is not always insurance against error. In this dis- 
cussion of mind, there are places where the reader will be made uncom- 
fortable by what appears to be a gross carelessness with words. Perhaps 
in a lecture, some license must be allowed for tue use of metaphor, but 
metaphors may cease to be figures of speech and become ‘‘things’’ them- 
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selves, and they do here. Mind becomes as solid seeming substance as any 
body, as real a thing as any medieval soul ‘‘ (In short, under the rubric 
‘pain’ we meet mind moving matter to help mind in mind’s distress,’’ 
p- 287), though indeed, in more meticulous moments, Sherrington is 
aware of the dangers of reification and warns against them. He is a 
dualist, recognizing a world made up of energy and of mind, yet insistent 
on their unity in the organism. ‘‘The mind and body are two series of 
events concomitant in time as parts of a single series.’’ (p. 354.) 

The last chapter in the book, in which Sherrington sets forth his hopes 
for mankind, seems a mixture of solid thinking with hopeful thinking— 
too sanguine in spots, perhaps, yet stoical too. Man’s struggle against 
nature, against man, and against himself is a theme ‘‘not unworthy the 
epic and the lyric’’—nor of the author’s own efforts to add to our under- 
standing of the struggle. 

Sherrington’s discussion of mind brings up an important issue, an issue 
of no ephemeral sort. There will be more Gifford lectures, other scien- 
tists and thinkers who will attack problems at least involving psychology, 
each from his own frame of reference. Their contribution to psychologi- 
eal thought should be vigorous and stimulating. But it will be so only if 
they are aware of the experimentally determined facts of modern psy- 
chology. To ignore these facts is certainly brash, and results are not 
inspiring. Sherrington seems to know Pavlov, Freud, Lashley, Charles 
Myers, and a bit of the ‘‘German figurists,’’ but to be relatively unin- 
formed with regard to the extensive work more recently done in condition- 
ing, in motor theory, in brain extirpation, in psychotherapy. What 
psychologist would have the temerity to write a book involving much 
biology guided only by Darwin, Huxley, Spencer, and Pasteur? The 
time has come when not only metaphysical speculation but, indeed, 
physiological theory as well, must square with psychological facts. 

NICHOLAS Hosss, 
The Citadel, 
Charleston, S.C. 


STarcH, DANIEL, STANTON, HAzEL, and KoerTH, WILHELMINE. Psychol- 
agy in Education. New York, D. Appleton-Century Company, 1941. 
Pp. 722. 

**Psychology must be the handmaiden of the teacher in his relations with 
pupils, colleagues, and citizens of the community. The object of this book 
is to bring to the desk of the teacher what psychology has to offer to help 
him or her to be a better teacher.’’ To avoid needless repetition, ‘‘the 
basie facts regarding learning, remembering, thinking, personality, study 
habits, and the like were discussed in ‘Controlling Human Behavior’ 
. « «’’ but specific reference is made to specific page numbers. Evidently 
the two books might be used together. 
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Although Starch conceived and outlined the book, each author is respon- 
sible for certain chapters. Experiences of the two co-authors which 
prepared them for writing most of the book are described on p. vi: ‘‘ Be- 
fore writing this book, Dr. Stanton and Dr. Koerth had the privilege of 
interviewing more than two hundred educators and psychologists *:vm 
the western to the eastern seaboard of the United States; the advantage 
of observing school practices in a wide variety of elementary, junior, and 
senior high schools and the cooperation of administrators and eachers in 
seeing reports of concrete teaching and learning situations and other 
valuable materials.’’ Throughout, the book gives evidence of its back- 
ground. It is a well organized and neatly arranged enumeration or com- 
pilation of psychological principles or factors which might be memori7~4d 
by the teacher and repeated in appropriate situations. Being in other 
work than teaching (psychology), the authors display a remarkable 
alertness in psychological literature. 

Part I of the book deals with Directing Pupils in School Environment. 
Emphasis is given to understanding young people, motivating them, and 
measuring their progress and achievement. Under the subtitle, Minimum 
Reading Knowledge of Statistics, p. 226, are references to statistic books, 
then follow such terms as Frequency, Rank order, Mean, Median, Range, 
Average Deviation, Standard Deviation, ete. Each term is defined suc- 
cinctly in a few lines but there is no illustration of their meanings in rela- 
tion to test scores or a problem. This is a typical attempt in this book to 
make teachers proficient. 

Part II of the book is titled ‘‘Guiding Pupils in Learning Specific Sub- 
jects.’’ Each of nine chapters discusses specific conrses in high school. 
The general tone may be represented by a few sentences: (p. 483) 
‘*Learning and training require both time and persistent effort’’; (p. 
484) ‘‘Proficiency in the language arts is vital in student success in the 
social sciences’’; (p. 486) ‘‘ Testing skills in the social sciences is still far 
from an exact science’’; (p. 489) ‘‘Learning to cope with ideas is per- 
haps the most important outcome of education.’’ 

The style is facile and it would seem that most of the material is quite 
suitable for talks to such groups as Parent-Teacher Associations. 

J. R. GENTRY, 
Ohio University 
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